mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-12-27 15:08:43 +00:00
8af0d816e6
* bug: fix the date_fields request bottleneck I encountered a performance issue while attempting to index 1 million vectors. Despite the Weaviate instance having low utilization, the process was estimated to take around 10 hours. After some investigation, I identified the bottleneck: _get_date_properties function was being called for every document, consequently a request to the Weaviate client was being sent and awaited for each document. To address this, I optimized the code by invoking the _get_date_properties function only when there is a schema change. This modification resulted in a notable performance improvement, reducing the indexing time to approximately 90 minutes for the same 1 million vectors. * bug: fix the date_fields request bottleneck * fix: executed the pre commit hooks for #9341
5 lines
104 B
YAML
5 lines
104 B
YAML
---
|
|
fixes:
|
|
- |
|
|
Fixed a bottleneck in Weaviate document store which was slowing down the indexing.
|