* add filter classes
* update filter comments
* Add util classes for converting filters (#2123)
* Apply Black
* reintroduce eval functions to filter ops
* Update documentation
* update to latest pymilvus version
* Apply Black
* fixing type hints
* Apply Black
* update write_documents method of milvus2 doc store
* remove unnecessary method
* update init
* remove changes to milvus 2 as they are part of other PR
* remove changes to milvus 2 as they are part of other PR
* updating doc strings to match elastic search filter doc
* Update Documentation & Code Style
* add support for case where there is no meta data defined for key
* update behaviour in case of field not existing in entry
* Update Documentation & Code Style
* add test for InMemoryDocumentStore extended meta data filtering
* make type hint more precise
Co-authored-by: bogdankostic <bogdankostic@web.de>
Co-authored-by: Sara Zan <sara.zanzottera@deepset.ai>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* Add versions of the the openapi.json file for versioned documentation
* Update Documentation & Code Style
* Use .suffix for extension check
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* update remaining occurences of get_connection
* first commit to add extended metadata filtering support to sql
* fix bugs
* adding sql doc store instead of milvus
* removing updates to milvus2 from other PR
* fixing not operator
* delete left over line
* remove unnecessary import
* Update Documentation & Code Style
* fix circular import
* fix left over merge conflict
* Update Documentation & Code Style
* fix abstract class
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Remove invalid-envvar-default and logging-too-many-args
* Remove import-self, access-member-before-definition and deprecated-argument
* Remove used-before-assignment by restructuring type import
* Remove unneeded-not
* Silence unnecessary-lambda (it's necessary)
* Remove pointless-string-statement
* Update Documentation & Code Style
* Silenced unsupported-membership-test (probably a real bug, can't fix though)
* Remove trailing-newlines
* Remove super-init-not-called and slience invalid-sequence-index (it's valid)
* Remove invalid-envvar-default in ui
* Remove some more warnings from pyproject.toml than actually solrted in code, CI will fail
* Linting all modules together is more readable
* Update Documentation & Code Style
* Typo in pylint disable comment
* Simplify long boolean statement
* Simplify init call in FAISS
* Fix inconsistent-return-statements
* Fix useless-super-delegation
* Fix useless-else-on-loop
* Fix another inconsistent-return-statements
* Move back pylint disable comment moved by black
* Fix consider-using-set-comprehension
* Fix another consider-using-set-comprehension
* Silence non-parent-init-called
* Update pylint exclusion list
* Update Documentation & Code Style
* Resolve unnecessary-else-after-break
* Fix superfluous-parens
* Fix no-else-break
* Remove is_correctly_retrieved along with its pylint issue
* Update exclusions list
* Silence constructor issue in squad_data.py (method is already broken)
* Fix too-many-return-statements
* Fix use-dict-literal
* Fix consider-using-from-import and useless-object-inheritance
* Update exclusion list
* Fix simplifiable-if-statements
* Fix one consider-using-dict-items
* Fix another consider-using-dict-items
* Fix a third consider-using-dict-items
* Fix last consider-using-dict-items
* Fix three use-a-generator
* Silence import errors on numba, tensorboardX and apex, but add comments & logs
* Fix couple of mypy issues
* Fix another typing issue
* Silence mypy, was conflicting with more meaningful pylint issue
* Fix no-else-continue
* Silence unsubscriptable-object and fix an import error with importlib.metadata
* Update Documentation & Code Style
* Fix all no-else-raise
* Update Documentation & Code Style
* Fix inverted parameters in simplified if switch
* Change [test] to [all] in some jobs (for typing and linting)
* Add comment in haystack/schema.py on pydantic's dataclasses
* Move comment from get_documents_by_id into _convert_weaviate_result_to_document in weaviate.py
* Add comment on pylint silencing
* Fix bug introduced rest_api/controller/search.py
* Update Documentation & Code Style
* Add ADR about Pydantic dataclasses
* Update pydantic-dataclasses.md
* Add link to Pydantic docs on Dataclasses
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Make YAML files get the same version as Haystack and throw warning at load in case of mismatch
* Update version of most YAMLs in the codebase (aesthethic chamge, only to avoid the warning).
* Remove quotes from version in tests
* Fix version in generate_json_schema.py
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Add proper annotation to MultiLabel and slightly improve __str__ for MultiLabel and Label
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Trigger Black (@ZanSara on external account)
* Trigger Black again
* Update Documentation & Code Style
* Add image of how it looks from the contributor's side
* Add screenshot of the check to enable maintainer's access to the fork
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: ZanSara <sarazanzo94@gmail.com>
* Add explicit reference to repo name to allow CI to push code back
* Run test matrix only on tested code changes
* Isolate the bot to check if it works
* Clarify situation with a comment
* Simplify autoformat.yml
* Add code and docs check
* Add git pull to make sure to fetch changes if they were created
* Add cache to autoformat.yml too
* Add information on forks in CONTRIBUTING.md
* Add a not about code quality tools in CONTRIBUTING.md
* Add image file types to the CI exclusion list
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Handle no answers and no documents scenarios in '_process_request'
* Fix tests
* Change return type in '_process_request'
* Return to use dicts
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Add type check for meta & add tests
* Improve tests
* Handle properly the ValueError ad an HTTPException
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Apply filter in eval only if no gold documents are given as input
* change type annotation of input documents in eval
* Update Documentation & Code Style
* fix mypy
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Add DELETE /feedback for testing and make the ID generate server-side
* Make sure to delete only user generated labels
* Reduce fixture scope, was too broad
* Make test a bit more generic
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* join node should allow reciprocal rank fusion
* Update Documentation & Code Style
* add missing merging mode
* tuples are immutable
* take correct results from pipeline
* Update Documentation & Code Style
* Simple docstrings, use ValueError
* Use K=60
* Minor refactoring
* precalculate expected result in test
* Update Documentation & Code Style
* refactor to make more clear
* rm unused imports
* tests should test only one thing
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: dmigo <d.f.goryunov@gmail.com>
* Fix dependency issue in Tutorial 6
* Remove faiss from first install block
* move faiss group back to main installation step
* Comment out Milvus cell
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* move commandline args to global conftest
* correct test exclude paths
* Update Documentation & Code Style
* exclude test_generator_pipeline_with_translator from windows ci
* exclude further oom tests
* enable log_cli
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Make mypy check also ui and rest_api, fix ui
* Remove explicit type packages from extras, mypy now downloads them
* Make pylint and mypy run on every file except tests
* Rename tasks
* Change cache key
* Fix mypy errors in rest_api
* Normalize python versions to avoid cache misses
* Add all exclusions to make pylint pass
* Run mypy on rest_api and ui as well
* test if installing the package really changes outcome
* Comment out installation of packages
* Experiment: randomize tests
* Add fallback installation steps on cache misses
* Remove randomization
* Add comment on cache
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Fix a path issue in Dockerfile-GPU
* Fix paths in Dockerfile-GPU
* Add workflow_dispatch to docker build task
* Remove reference to optional component from ui/, not needed anymore
* Move pytorch installation last to avoid replacing it later
* Remove optional import from rest_api too, no more needed
* Change path in ui/Dockerfile
* ui container works again
* Complete review of import paths
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* pass documents as extra param to eval
* pass documents via labels to eval
* rename param in docs
* Update Documentation & Code Style
* Revert "rename param in docs"
This reverts commit 2f4c2ec79575e9dd33a8300785f789a327df36f4.
* Revert "pass documents via labels to eval"
This reverts commit dcc51e41f2637d093d81c7d193b873c17c36b174.
* simplify iterating through labels and docs
* Update Documentation & Code Style
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
