yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-12-13 07:47:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

test: add meta fields for meta_config to be used during testing (#3021)

Browse Source 
* added meta fields for meta_config to be used during realtime testing of PineconeDocumentStore

* Add documentation on metadata filtering in  docstring

* docs

Co-authored-by: Sara Zan <sara.zanzottera@deepset.ai>
This commit is contained in:
James Briggs 2022-08-12 09:27:56 +01:00 committed by GitHub
parent 81a5949103
commit 26c938a8e6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 17 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/_src/api/api/document_store.md
View File

@ -4744,6 +4744,7 @@ lost if you choose to recreate the index. Be aware that both the document_index
be recreated. be recreated.
- `metadata_config`: Which metadata fields should be indexed. Should be in the format - `metadata_config`: Which metadata fields should be indexed. Should be in the format
`{"indexed": ["metadata-field-1", "metadata-field-2", "metadata-field-n"]}`. `{"indexed": ["metadata-field-1", "metadata-field-2", "metadata-field-n"]}`.
Indexing metadata fields is a prerequisite to allow filtering of documents by metadata values.
- `validate_index_sync`: Whether to check that the document count equals the embedding count at initialization time - `validate_index_sync`: Whether to check that the document count equals the embedding count at initialization time
<a id="pinecone.PineconeDocumentStore.write_documents"></a> <a id="pinecone.PineconeDocumentStore.write_documents"></a>

1
haystack/document_stores/pinecone.py
View File

@ -89,6 +89,7 @@ class PineconeDocumentStore(SQLDocumentStore):
be recreated. be recreated.
:param metadata_config: Which metadata fields should be indexed. Should be in the format :param metadata_config: Which metadata fields should be indexed. Should be in the format
`{"indexed": ["metadata-field-1", "metadata-field-2", "metadata-field-n"]}`. `{"indexed": ["metadata-field-1", "metadata-field-2", "metadata-field-n"]}`.
Indexing metadata fields is a prerequisite to allow filtering of documents by metadata values.
:param validate_index_sync: Whether to check that the document count equals the embedding count at initialization time :param validate_index_sync: Whether to check that the document count equals the embedding count at initialization time
""" """
# Connect to Pinecone server using python client binding # Connect to Pinecone server using python client binding

15
test/conftest.py
View File

@ -87,6 +87,20 @@ DC_TEST_INDEX = "document_retrieval_1"
DC_API_KEY = "NO_KEY" DC_API_KEY = "NO_KEY"
MOCK_DC = True MOCK_DC = True
# Set metadata fields used during testing for PineconeDocumentStore meta_config
META_FIELDS = [
"meta_field",
"name",
"date_field",
"numeric_field",
"f1",
"f3",
"meta_id",
"meta_field_for_count",
"meta_key_1",
"meta_key_2",
]
# Disable telemetry reports when running tests # Disable telemetry reports when running tests
posthog.disabled = True posthog.disabled = True
@ -979,6 +993,7 @@ def get_document_store(
index=index, index=index,
similarity=similarity, similarity=similarity,
recreate_index=recreate_index, recreate_index=recreate_index,
metadata_config={"indexed": META_FIELDS},
) )
else: else: