yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-12-13 07:47:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,115 Commits 100 Branches 267 Tags
Commit Graph

3115 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Bishal gaire
4198dc6feb
Update docstring for RAG (#1149) 
* Update 7.md

Initialize retriever in RAG generator

* update docstring

* Update 7.md

* Update 7.md

Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
 2021-08-09 11:52:45 +02:00
Malte Pietsch
66b10a508b
Update TOC of readme 2021-08-09 11:40:20 +02:00
Malte Pietsch
fb4d6e0381
Update README.md 2021-08-09 11:25:47 +02:00
Malte Pietsch
5a3ea5843f
Fix Tutorial Links 2021-08-09 11:22:19 +02:00
Shahrukh Khan
f99c14268a
Update README.md for new tutorials 13 and 14 (#1325) 
* Update README.md

* Update README.md
 2021-08-09 10:44:42 +02:00
Shahrukh Khan
cc43502e7e
Add Tutorial about Query Classifier (#1324) 
* add query classifier colab and jupyter notebook

* Delete Tutorial13_Query_Classifier.ipynb

* add query classifier tutorial with updated number

* add query classifier tutorial script

* Rename tutorial14_query_classifier.py to Tutorial14_Query_Classifier.py
 2021-08-09 10:43:50 +02:00
Markus Sagen
60cce4e579
Allow to upload multiple files simultaneously in Haystack UI (#1323) 
* Fix only one file upload for Haystack UI

When using Haystack UI and streamlit, the default behavior is to upload one file at at time and override the file you have already uploaded. Streamlit now supports uploading multiple files and may be more intuitive for users of Haystack to use it as the default behavior.

Return type for `st.sidebar.file_uploader` when `accept_multiple_files=True` is a list of the files and empty if no files are provided

* Update requirements.txt

* Update requirements.txt

Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
 2021-08-06 17:45:36 +02:00
David Silva
fb4a417e53
Integrate filters with knn queries in OpenDistroElasticsearchDocumentStore (#1301) 
* Integrate filters with knn queries in ODFE

Allows the use of filters coupled with knn similarity search on
OpenDistroElasticsearchDocumentStore instances. Fixes #1139.

* Satisfy mypy

Co-authored-by: David Silva <44328951+DavidSilva98@users.noreply.github.com>
Co-authored-by: brandenchan <brandenchan@icloud.com>
 2021-08-03 16:31:13 +02:00
Ikram Ali
6829446b0e
Transformer summarizer truncation bug fixed (#1309) 
* [Summarizer] truncation bug fixed. fix #1296

* [Summarizer] Fixes #1296

* [Summarizer] warning added Fixes #1296

* [Summarizer] code refactor. Fixes #1296

* Avoid repeating warning message

* Add type check

* Format string

Co-authored-by: brandenchan <brandenchan@icloud.com>
 2021-08-02 18:02:39 +02:00
oryx1729
bafa1b46de
Add Ray integration for Pipelines (#1255) 2021-08-02 14:51:24 +02:00
oryx1729
3eaf9dfbca
Suppress FAISS logs & apex warnings (#1315) 2021-07-29 14:32:50 +02:00
oryx1729
849cf27e74
Pin Weaviate version (#1306) 2021-07-27 16:44:07 +02:00
Branden Chan
937247d628
Add QuestionGenerator (#1267) 
* Create basic Question Generation

* Split texts into 50 word chunks

* Allow prompt to be changed

* Implement iteration functionality in DS

* Add docstrings, create pipelines

* Make pipelines work

* Add comments

* Add tests

* Add tutorials and docs

* Add doc string
 2021-07-26 17:20:43 +02:00
Branden Chan
363be65a78
Implement OpenSearch ANN (#1225) 
* Simplify ODES init

* Add arguments to ES init and create script

* Rename similarity_fn_name and add util fn

* Create OpenSearchDocumentStore

* Specify params of Open Search HNSW

* Add better argument handling

* Update opensearch index mapping

* Edit opensearch default port

* Fix HNSW mapping

* Force small HNSW params

* Implement auto start and stopping of document store services

* Fix starting and stopping of ds service

* Restore HNSW params

* Add opensearch query benchmarks

* Add write wait time

* Revert wait time

* Add timeout

* Update benchmarks

* Update benchmarks

* Update benchmarks json

* Update documentation

* Update documentation

* Fix similarity name

* Improve argument passing

* Improve stopping and starting of service
 2021-07-26 10:52:52 +02:00
Malte Pietsch
4c2a0b914a
Remove pipeline eval example script (#1297) 2021-07-21 11:12:04 +02:00
Srevin Saju
7d6548100a
Add support for elasticsearch to connect without any authentication (#1294) 2021-07-21 10:47:52 +02:00
oryx1729
e857233313
Add Header in sample REST API Search Request (#1293) 2021-07-19 12:57:43 +02:00
oryx1729
3f58d4c13b
Fix SQLAlchemy relationship warnings (#1289) 2021-07-15 17:59:59 +02:00
Bob van Luijt
8dae844447
Bump Weaviate version to 1.5 (#1287) 
* bump Weaviate version to 1.5

* bump Weaviate version to 1.5
 2021-07-15 08:26:22 +02:00
Ikram Ali
97c1e2cc90
[document_store] Raise warning when labels are overwritten (#1257) 
* [document_store]SQLDocumentStore write_labels() overwrite warning added.

* [document_store]SQLDocumentStore write_labels() overwrite warning added.

* [document_store] bug fixed. #1140

* [document_store] bug fixed. #1140

* [document_store] get_labels_by_id() method removed. #1140

* [document_store] Code refactor. fix #1140

* [document_store] Code refactor. fix #1140

* [document_store] elasticsearch document store Code refactor. fix #1140

* [document_store] elasticsearch document store Code refactor. fix #1140

* [document_store] elasticsearch document store Code refactor. fix #1140

* [document_store] Code refactor for better visibility. fix #1140

* [document_store] Inmemory document store duplicate labels warning added fix #1140
 2021-07-14 16:21:04 +02:00
Branden Chan
da97d81305
Change variable names (#1286) 2021-07-14 14:03:34 +02:00
Branden Chan
7717e81ecc
Improve preprocessing logging (#1263) 
* Improve preprocessing logging

* Change variable names

* Change variable names

* Satisfy mypy
 2021-07-14 14:03:13 +02:00
oryx1729
c318b5853b
Serialize crawler output to JSON (#1284) 2021-07-14 13:16:27 +02:00
Julian Risch
4e6f7f349d
Add FARMClassifier node for Document Classification (#1265) 
* Add FARM classification node

* Add classification output to meta field of document

* Update usage example

* Add test case for FARMClassifier

* Replace FARMRanker with FARMClassifier in documentation strings

* Remove base method not implemented by any child class, etc.
 2021-07-13 21:44:26 +02:00
Antonio De Marinis
f79d9bdca6
Upgrade streamlit and adjust height of result texts dynamically (#1279) 
* update to latest streamlit and st-annotated-text

* improve ui results by passing dynamic height to annotated-text
 2021-07-13 18:59:39 +02:00
threepointsomeone
2f93c2ddd5
Added explicit refresh call during refresh_type is false in update embedding. (#1259) 
Co-authored-by: vishwaspai <vishwas.pai@emplay.net>
 2021-07-13 16:59:09 +02:00
Julian Risch
90f826e95e
Add links to tutorial 12 to readme (#1274) 2021-07-13 11:23:10 +02:00
Julian Risch
2a90471c73
Encapsulate tutorial code in method (#1266) 2021-07-09 17:08:19 +02:00
Julian Risch
dbb9efbd39
Add SentenceTransformersRanker with pre-trained Cross-Encoder (#1209) 
* Add SentenceTransformersRanker with pre-trained Cross-Encoder

* Add test cases for Ranker nodes and update documentation

* update docstring

* Update docstring

* Update __init__.py

* update import for test

Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
 2021-07-07 17:31:45 +02:00
Moshe Berchansky
495f98deba
Add global_loss_buffer_size to the DensePassageRetriever, in order to fix 'encoded data exceeds max_size' error with DDP. (#1245) 2021-07-06 13:56:41 +02:00
Ikram Ali
f5a8d3cf45
Add id in write_labels() for SQLDocumentStore (#1253) 2021-07-05 14:13:21 +02:00
Ikram Ali
8e117f5e11
ElasticsearchDocumentStore get_label_count() bug fixed. (#1252) 2021-07-03 20:51:33 +02:00
Ikram Ali
04a470f890
SQLDocumentStore get_label_count() bug fixed. (#1251) 2021-07-03 14:02:44 +02:00
Michaël Bitard
aaed22304d
Fix convert integer CONCURRENT_REQUEST_PER_WORKER (#1247) 2021-07-02 20:38:15 +02:00
Ikram Ali
29e140196b
[pipeline] Allow for batch indexing when using Pipelines fix #1168 (#1231) 
* [pipeline] Allow for batch indexing when using Pipelines fix #1168

* [pipeline] Test case fixed fix #1168

* [file_converter] Path.suffix updated #1168

* [file_converter] meta can be one of these three cases:
                 A single dict that is applied to all files
                 One dict for each file being converted
                 None #1168

* [file_converter] mypy error fixed.

* [file_converter] mypy error fixed.

* [rest_api] batch file upload introduced in indexing API.

* [test_case] Test_api file upload parameter name updated.

* [ui] Streamlit file upload parameter updated.
 2021-06-30 14:13:46 +02:00
Malte Pietsch
5e23e72f31 Update issue templates 2021-06-30 12:12:07 +02:00
Guillim
73a4f9825a
Add env var CONCURRENT_REQUEST_PER_WORKER (#1235) 
* we create an env var `CONCURRENT_REQUEST_PER_WORKER` following your naming convention, (I came a few commit backwards to find the original name)

* default to 4
 2021-06-29 07:44:25 +02:00
Malte Pietsch
2c964db62d
Relax typing for meta data in REST API (#1224) 2021-06-24 12:34:42 +02:00
Malte Pietsch
2caeea000e
Small UI and REST API fixes (#1223) 
* small fixes

* change default question
 2021-06-24 09:53:08 +02:00
Julian Risch
17dcb8c23e
Use Reader's device by default (#1208) 
* Use Reader's device by default

* Replace get_device with initialize_device_settings

* Add import statements for init_device_settings

* Remove unused get_device method
 2021-06-24 09:22:34 +02:00
Branden Chan
10e332dabb
Fix Links (#1199) 
* Fix link highlight

* Regen md files

* Remove duplicate

* Fix whitespace

* fixing strings for website

* Fix link

Co-authored-by: PiffPaffM <markuspaff.mp@gmail.com>
 2021-06-23 19:07:54 +02:00
Branden Chan
efc03f72db
Make PreProcessor.process() work on lists of documents (#1163) 
* Add process_batch method

* Rename methods

* Fix doc string, satisfy mypy

* Fix mypy CI

* Fix typp

* Update tutorial

* Fix argument name

* Change arg name

* Incorporate reviewer feedback
 2021-06-23 18:13:51 +02:00
oryx1729
afee4f36ce
Add scaffold for defining custom components for Pipelines (#1205) 2021-06-23 12:01:54 +02:00
vblagoje
02fc4c7783
Improve document stores unit test parametrization (#1202) 2021-06-22 16:08:23 +02:00
Markus Paff
a8f3601e6a
Pin docs for 0.9.0 2021-06-22 10:38:08 +02:00
Ikram Ali
d835a9cdc5
[setup] version tag added to Haystack fix #1175 (#1216) 2021-06-22 09:43:26 +02:00
Stefano
66049abff0
Add arg to support different languages in PreProcessor's sentence segmentation (#1160) 
* Add PreProcessor optional language parameter.

* Add iso639 to nltk languages.

* Update docstring

Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
 2021-06-21 18:53:19 +02:00
Julian Risch
9e4d7bf9be
Increase Haystack version to 0.9.0 (#1215) v0.9.0 2021-06-21 18:39:00 +02:00
oryx1729
0168f04385
Remove unused function _get_pseudo_prob (#1201) 2021-06-17 10:28:48 +02:00
C V Goudar
f9c4083006
Bugfix setting of device by defaulting to "cpu" (#1182) 
* Defaulting the device to cpu in case gpu is not available and use_gpu is set to True

Co-authored-by: C V Goudar <cv.goudar@emplay.et>
 2021-06-16 10:26:29 +02:00