Ivana Zeljkovic
2326f2f9fe
feat: Pinecone document store optimizations ( #5902 )
...
* Optimize methods for deleting documents and getting vector count. Enable warning messages when Pinecone limits are exceeded on Starter index type.
* Fix typo
* Add release note
* Fix mypy errors
* Remove unused import. Fix warning logging message.
* Update release note with description about limits for Starter index type in Pinecone
* Improve code base by:
- Adding new test cases for get_embedding_count method
- Fixing get_embedding_count method
- Improving delete documents
- Fix label retrieval
- Increase default batch size
- Improve get_document_count method
* Remove unused variable
* Fix mypy issues
2023-10-16 19:26:24 +02:00
Christian Clauss
30ca042370
ci: Use ruff in pre-commit to further limit code complexity ( #5783 )
...
* ci: Use ruff in pre-commit to further limit complexity
* Delete releasenotes/notes/ruff-4d2504d362035166.yaml
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2023-09-13 15:18:16 +02:00
Silvano Cerza
274746db07
style: Update black ( #4101 )
...
* Update black version
* Format file with new black style
* Update black pre-commit hook version
2023-02-08 15:34:43 +01:00
tstadel
600dc2d611
refactor: filters type ( #3682 )
...
* consolidate filters type
* remove unnecessary optionals
* fix mypy
* fix pylint
* fix pylint
* move FilterType to schema
* remove Optional from FilterType
* move to Dict[str, Any]
* Revert "move to Dict[str, Any]"
This reverts commit e8c561bb7885949e19825697fa4c469945f90ce5.
* fix mypy
* fix pylint
* revert isort changes in elasticsearch
* remove todos in milvus.py
* remove todos in sql.py
* add aggregate_labels tests
* consolidate aggregate_labels tests
* remove superfluous type todos
* remove ALL superfluous #todos
2022-12-12 14:04:29 +01:00
Sara Zan
dcb132ba59
chore: remove f-strings from logs for performance reasons ( #3212 )
...
* Use the %s syntax on all debug messages
* Use the %s syntax on some more debug messages
* Use the %s syntax on info messages
* Use the %s syntax on warning messages
* Use the %s syntax on error and exception messages
* mypy
* pylint
* trogger tutorials execution in CI
* trigger tutorials execution on CI
* black
* remove embeddings from repr
* fix Document `__repr__`
* address feedback
* mypy
2022-09-19 18:18:32 +02:00
James Briggs
9b1b03002f
update to PineconeDocumentStore to remove dependency on SQL db ( #2749 )
...
* update to PineconeDocumentStore to remove dependency on SQL db
* Update Documentation & Code Style
* typing fixes
* Update Documentation & Code Style
* fixed embedding generator to yield Documents
* Update Documentation & Code Style
* fixes for final typing issues
* fixes for pylint
* Update Documentation & Code Style
* uncomment pinecone tests
* added new params to docstrings
* Update Documentation & Code Style
* Update Documentation & Code Style
* Update haystack/document_stores/pinecone.py
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* Update haystack/document_stores/pinecone.py
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* Update Documentation & Code Style
* Update haystack/document_stores/pinecone.py
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* Update haystack/document_stores/pinecone.py
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* Update haystack/document_stores/pinecone.py
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* Update haystack/document_stores/pinecone.py
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
* changes based on comments, updated errors and install
* Update Documentation & Code Style
* mypy
* implement simple filtering in pinecone mock
* typo
* typo in reverse
* account for missing meta key in filtering
* typo
* added metadata filtering to describe index
* added handling for users switching indexes in same doc store, and handling duplicate docs in write
* syntax tweaks
* added index option to document/embedding count calls
* labels implementation in progress
* added metadata fields to be indexed for pinecone tests
* further changes to mock
* WIP implementation of labels+multilabels
* switched to rely on labels namespace rather than filter
* simpler delete_labels
* label fixes, remove debug code
* Apply dostring fixes
Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com>
* mypy
* pylint
* docs
* temporarily un-mock Pinecone
* Small Pinecone test suite
* pylint
* Add fake test key to pass the None check
* Add again fake test key to pass the None check
* Add Pinecone to default docstores and fix filters
* Fix field name
* Change field name
* Change field value
* Remove comments
* forgot to upgrade pyproject.toml
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sara Zan <sara.zanzottera@deepset.ai>
Co-authored-by: Sara Zan <sarazanzo94@gmail.com>
Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com>
2022-08-24 13:27:15 +02:00
James Briggs
82c9cff3d9
test: update filtering of Pinecone mock to imitate doc store ( #3020 )
...
* updated filtering of doc store to imitate pinecone
* Update test/mocks/pinecone.py
2022-08-18 09:57:08 +02:00
James Briggs
a4e197c21a
changed mock pinecone to use dict rather than list index ( #2845 )
2022-07-19 15:28:22 +02:00
Sara Zan
6b39fbd39c
Mocking Pinecone tests ( #2778 )
...
* Integrating the mock into conftest.py
* re-enable workflow
* delete_all
* Update Documentation & Code Style
* remove ValueError
* Add empty response
* wrong condition
* return response
* revert removal of delete_all
* change mock
* Update Documentation & Code Style
* test for rest api, to revert
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-14 20:03:33 +02:00
