* Add tests for missing init and super
* Update Documentation & Code Style
* change in with endswith
* Move test in pipeline.py and change test in pipeline_yaml.py
* Update Documentation & Code Style
* Use caplog to test the warning
* Update Documentation & Code Style
* move tests into test_pipeline and use get_config
* Update Documentation & Code Style
* Unmock version name
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* remove duplicate imports
* fix ungrouped-imports
* Fix wrong-import-position
* Fix unused-import
* pyproject.toml
* Working on wrong-import-order
* Solve wrong-import-order
* fix Pool import
* Move open_search_index_to_document_store and elasticsearch_index_to_document_store in elasticsearch.py
* remove Converter from modeling
* Fix mypy issues on adaptive_model.py
* create es_converter.py
* remove converter import
* change import path in tests
* Restructure REST API to not rely on global vars from search.apy and improve tests
* Fix openapi generator
* Move variable initialization
* Change type of FilterRequest.filters
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Delete files in _src
* Filter unused images and re-add images that were in use in docs/img
* Remove all usages of user-images.githubusercontent.com
Co-authored-by: ZanSara <sarazanzo94@gmail.com>
* Make initialize_device_settings take a devices list, and change signature of FARMReader
* reintroduce use_gpu and propagate devices to other methods
* fix typing for initialize_device_settings
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ensure correct embedding_encoder is loaded when embedding_model is a sentence-transformers model but model_format is missing or wrong
* minor refactoring
* do not update model_format and ensure a warning is logged when it could be wrong
* Apply black
* Apply black
Co-authored-by: Michele Pangrazzi <michele@wonderflow.ai>
Co-authored-by: bogdankostic <bogdankostic@web.de>
* extract extension based on file's content
* Add python-magic dependency
* fix the _estimate_extension function and lowercase the file extensions
* check if the FileTypeClassifier can be imported
* add test and new file types
* fix typing
* import Optional
* revert Optional and make sure a string is always returned
* fix test so that it skips markdown files
* Emulate Code & Docs action
* Generate schemas
* Tidy up test code & extensioness files
* Improve error messages
* Revert schema changes
* Emulate black and docs CI again
* Fix 'bug' on Weaviate only returning max. 100 docs on get_all_documents
* Add type
* Update Weaviate version on the CI
* Fix bug on get_document_count where there are no documents
* Add more info in the docstrings of get_all_documents and get_all_documents_generator
* Add latest docstring and tutorial changes
* Apply Black
* Update Documentation & Code Style
* Trigger pipeline
* Update Documentation & Code Style
* Include StefanBogdan feedback
* Fix mypy issues and LogicalFilterClause
* Add more types
* Update Documentation & Code Style
* update setup.cfg
* Upgrade weaviate containers too
* Allow to filter for content field in Weaviate
* Use convert_to_weaviate instead of convert_to_pinecone
* Fix _get_all_documents_in_index
* Update docstrings and docs
* Catching an exception in get_document(s)_by_id
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bogdankostic <bogdankostic@web.de>
* EvaluationSetClient for deepset cloud to fetch evaluation sets and labels for one specific evaluation set
* make DeepsetCloudDocumentStore able to fetch uploaded evaluation set names
* fix missing renaming of get_evaluation_set_names in DeepsetCloudDocumentStore
* update documentation for evaluation set functionality in deepset cloud document store
* DeepsetCloudDocumentStore tests for evaluation set functionality
* rename index to evaluation_set_name for DeepsetCloudDocumentStore evaluation set functionality
* raise DeepsetCloudError when no labels were found for evaluation set
* make use of .get_with_auto_paging in EvaluationSetClient
* Return result of get_with_auto_paging() as it parses the response already
* Make schema import source more specific
* fetch all evaluation sets for a workspace in deepset Cloud
* Rename evaluation_set_name to label_index
* make use of generator functionality for fetching labels
* Update Documentation & Code Style
* Adjust function input for DeepsetCloudDocumentStore.get_all_labels, adjust tests for it, fix typos, make linter happy
* Match error message with pytest.raises
* Update Documentation & Code Style
* DeepsetCloudDocumentStore.get_labels_count raises DeepsetCloudError when no evaluation set was found to count labels on
* remove unneeded import in tests
* DeepsetCloudDocumentStore tests, make reponse bodies a string through json.dumps
* DeepsetcloudDocumentStore.get_label_count - move raise to return
* stringify uuid before json.dump as uuid is not serilizable
* DeepsetcloudDocumentStore - adjust response mocking in tests
* DeepsetcloudDocumentStore - json dump response body in test
* DeepsetCloudDocumentStore introduce label_index, EvaluationSetClient rename label_index to evaluation_set
* Update Documentation & Code Style
* DeepsetCloudDocumentStore rename evaluation_set to evaluation_set_response as there is a name clash with the input variable
* DeepsetCloudDocumentStore - rename missed variable in test
* DeepsetCloudDocumentStore - rename missed label_index to index in doc string, rename label_index to evaluation_set in EvaluationSetClient
* Update Documentation & Code Style
* DeepsetCloudDocumentStore - update docstrings for EvaluationSetClient
* DeepsetCloudDocumentStore - fix typo in doc string
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* use "must" instead of "should" for query-matching
* Update Documentation & Code Style
* fix mypy issue
* fix finding of new pylint version
* add test
* fix test_retrieval
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* change docu text about how to opt-out
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Run Pinecone tests only if files related to Pinecone changed
* Change in pinecone.py that will be reverted
* Revert change in pinecone.py
* Test Pinecone also when filter_utils.py changes
* added core install and functionality of pinecone doc store (init, upsert, query, delete)
* implemented core functionality of Pinecone doc store
* Update Documentation & Code Style
* updated filtering to use Haystack filtering and reduced default batch_size
* Update Documentation & Code Style
* removed debugging code
* updated Pinecone filtering to use filter_utils
* removed uneeded methods and minor tweaks to current methods
* fixed typing issues
* Update Documentation & Code Style
* Allow filters in al methods except get_embedding_count
* Fix skipping document store tests
* Update Documentation & Code Style
* Fix handling of Milvus1 and Milvus2 in tests
* Update Documentation & Code Style
* Fix handling of Milvus1 and Milvus2 in tests
* Update Documentation & Code Style
* Remove SQL from tests requiring embeddings
* Update Documentation & Code Style
* Fix get_embedding_count of Milvus2
* Make sure to start Milvus2 tests with a new collection
* Add pinecone to test suite
* Update Documentation & Code Style
* Fix typing
* Update Documentation & Code Style
* Add pinecone to docstores dependendcy
* Add PineconeDocStore to API Documentation
* Add missing comma
* Update Documentation & Code Style
* Adapt format of doc strings
* Update Documentation & Code Style
* Set API key as environment variable
* Skip Pinecone tests in forks
* Add sleep after deleting index
* Add sleep after deleting index
* Add sleep after creating index
* Add check if index ready
* Remove printing of index stats
* Create new index for each pinecone test
* Use RestAPI instead of Python API for describe_index_stats
* Fix accessing describe_index_stats
* Remove usages of describe_index_stats
* Run pinecone tests separately
* Update Documentation & Code Style
* Add pdftotext to pinecone tests
* Remove sleep from doc store fixture
* Add describe_index_stats
* Remove unused imports
* Use pull_request_target trigger
* Revert use pull_request_target trigger
* Remove set_config
* Add os to conftest
* Integrate review comments
* Set include_values to False
* Remove quotation marks from pinecone.Index type
* Update Documentation & Code Style
* Update Documentation & Code Style
* Fix number of args in error messages
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bogdankostic <bogdankostic@web.de>
* Similar test case seems to pass
* Update Documentation & Code Style
* Improve error message
* Slightly clarify info message
* Fix mismatch between node and node_class in the schema generation
* Remove condition that node class names cannot begin with Base and update tests
* Indentation
* Update Documentation & Code Style
* feedback
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* add basic telemetry features
* change pipeline_config to _component_config
* Update Documentation & Code Style
* add super().__init__() calls to error classes
* make posthog mock work with python 3.7
* Update Documentation & Code Style
* update link to docs web page
* log exceptions, send event for raised HaystackErrors, refactor Path(CONFIG_PATH)
* add comment on send_event in BaseComponent.init() and fix mypy
* mock NonPrivateParameters and fix pylint undefined-variable
* Update Documentation & Code Style
* check model path contains multiple /
* add test for writing to file
* add test for en-/disable telemetry
* Update Documentation & Code Style
* merge file deletion methods and ignore pylint global statement
* Update Documentation & Code Style
* set env variable in demo to activate telemetry
* fix mock of HAYSTACK_TELEMETRY_ENABLED
* fix mypy and linter
* add CI as env variable to execution contexts
* remove threading, add test for custom error event
* Update Documentation & Code Style
* simplify config/log file deletion
* add test for final event being sent
* force writing config file in test
* make test compatible with python 3.7
* switch to posthog production server
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Add batch_size and use_gpu to SAS from #2306
* Add batch_size and use_gpu to SAS from #2306
* Added docstrings for SAS-GPU to evluator.py
* Added docstrings for SAS-GPU to pipelines/base.py
* Typo fix in pipelines/base.py
* streamline docstrings with related params in code base
Co-authored-by: Thomas Stadelmann <thomas.stadelmann@deepset.ai>
* 'os' wrapper to function for brownfield support
* Changing function names and fixing default parameter values
* Including parameter keys
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Restrain the files loaded by options.packages.find
* Move json-schemas inside haystack
* Change syntax in setup.cfg
* Change again syntax in options.package_data
* Specify more exactly which files to package
Co-authored-by: Thomas Stadelmann <thomas.stadelmann@deepset.ai>
