yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2026-02-02 21:13:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,166 Commits 108 Branches 273 Tags
Commit Graph

4166 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
David S. Batista
e349a7f2fc
docs: complete docstring for DocumentJoiner code example (#8593) 
* initial import

* changing a method to static

* reverting staticmethod
 2024-12-05 14:04:34 +00:00
David S. Batista
2282c26f17
feat!: SentenceWindowRetriever returns List[Document] with docs ordered by split_idx_start (#8590) 
* initial import

* adding a few pylint disable

* adding tests

* fixing integration tests

* adding release notes

* fixing types and docstrings
 2024-12-04 16:55:56 +01:00
David S. Batista
f0638b2868
refactor: moving SentenceSplitter outside NLTKDocumentSplitter (#8599) 
* initial import

* fixing imports and renaming file

* fixing imports path

* adding condition to check NLTK successfully imported

* adding one class inside the NLTK imported condition
 2024-12-04 10:44:36 +01:00
David S. Batista
c5ef0b2956
chore: adding a deprecation warning on the SentenceWindowRetriever (#8597) 
* linting

* improving message

* fixing header

* adding deprecation in the release notes
 2024-12-03 17:41:19 +01:00
Julian Risch
41369b9e0a
chore: Mention breaking changes in PR template (#8602) 2024-12-03 17:18:48 +01:00
Amna Mubashar
4c8eb54049
feat: Add store_full_path to converters (3/3) (#8585) 
* Add store_full_path params
 2024-12-03 13:48:56 +05:00
Stefano Fiorucci
de7099e560
ci: add job to check imports (#8594) 
* try checking imports

* clarify error message

* better fmt

* do not show complete list of successfully imported packages

* refinements

* relnote

* add missing forward references

* better function name

* linting

* fix linting

* Update .github/utils/check_imports.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-11-29 14:00:59 +00:00
Madeesh Kannan
163c06f3d6
chore: Revert change to deserialization error in Pipeline (#8591) 2024-11-28 13:28:52 +01:00
Stefano Fiorucci
c8685aa141
refactor: update components to access ChatMessage.text instead of content (#8589) 
* introduce text property and deprecate content

* release note

* use chatmessage.text

* release note

* linting
 2024-11-28 10:16:07 +00:00
Stefano Fiorucci
fb1baf4921
refactor: ChatMessage - introduce text property and deprecate content (#8588) 
* introduce text property and deprecate content

* release note

* minor test refactoring

---------

Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>
 2024-11-28 09:53:02 +00:00
Stefano Fiorucci
51c1390426
chore: use class methods to create ChatMessage (#8581) 
* use class methods to build messages

* fix failing format
 2024-11-28 09:35:24 +00:00
Silvano Cerza
473f7bef11 Change Pipeline.from_dict error message 2024-11-28 10:15:06 +01:00
Stefano Fiorucci
fb42c035c5
feat: PyPDFToDocument - add new customization parameters (#8574) 
* deprecat converter in pypdf

* fix linting of MetaFieldGroupingRanker

* linting

* pypdftodocument: add customization params

* fix mypy

* incorporate feedback
 2024-11-26 16:37:59 +01:00
Stefano Fiorucci
2440a5ee17
chore:PyPDFToDocument - deprecate converter init parameter (#8569) 
* deprecat converter in pypdf

* fix linting of MetaFieldGroupingRanker

* linting
 2024-11-26 14:47:04 +01:00
Haystack Bot
3d95e06722
chore: Update unstable version to 2.9.0-rc0 (#8582) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-11-26 16:42:12 +05:00
Michele Pangrazzi
f0c3692cf2
Remove is_greedy deprecated argument from @component decorator (#8580) 
* Remove 'is_greedy' deprecated argument from @component decorator

* Remove unused import
v2.9.0-rc0 		2024-11-26 10:44:50 +00:00
Vladimir Blagojevic
59f1e182db
feat: Add variable to specify inputs as optional to ConditionalRouter (#8568) 
* Add optional_variables in ConditionalRouter

* Add reno note

* Add more unit test with various complex scenarios

* Add more unit tests

* Add pylint disable=too-many-positional-arguments

* PR feedback from @sjrl
 2024-11-26 10:48:55 +01:00
Matt G
e3b73e048b
fix: bug on tracing where components are in a loop in a pipeline (#8576) 
* Fix to tracing parent spans on loops

* Fix linting

* Add release notes

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-11-25 14:21:08 +01:00
Silvano Cerza
ab840351f8
Fix DocumentCleaner not preserving Document fields (#8578) 2024-11-25 13:08:59 +01:00
Amna Mubashar
9302d3d9f0
feat: Add store_full_path to converters (2/3) (#8573) 2024-11-25 15:22:19 +05:00
Sebastian Husch Lee
eace2a99e5
feat: Add Literal["*"] option to required_variables in ChatPrompBuilder and PromptBuilder (#8572) 
* Add new option for required_variables in PromptBuilder and ChatPromptBuilder

* Add reno note

* Add tests
 2024-11-22 16:27:50 +01:00
David S. Batista
b5a2fad642
feat: adding Maximum Margin Relevance Ranker (#8554) 
* initial import

* linting

* adding MRR tests

* adding release notes

* fixing tests

* adding linting ignore to cross-encoder ranker

* update docstring

* refactoring

* making strategy Optional instead of Literal

* wip: adding unit tests

* refactoring MMR algorithm

* refactoring tests

* cleaning up and updating tests

* adding empty line between license + code

* bug in tests

* using Enum for strategy and similarity metric

* adding more tests

* adding empty line between license + code

* removing run time params

* PR comments

* PR comments

* fixing

* fixing serialisation

* fixing serialisation tests

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/sentence_transformers_diversity.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* fixing tests

* PR comments

* PR comments

* PR comments

* PR comments

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2024-11-22 14:58:45 +00:00
Richard Hudson
a8eeb2024f
feat: Allow unverified OpenAPI calls (#8562) 
* Feed through ssl_verify value to OpenAPI

* Add release note

* Update serialization methods

* Applied black formatting
 2024-11-22 15:45:00 +01:00
Amna Mubashar
4e6c7967d9
fix: Remove deprecated Legacy Filter Tests (#8567) 2024-11-22 14:39:22 +01:00
Amna Mubashar
21906d0558
feat: Add store_full_path to converters (1/3) (#8566) 
* Add store_full_path param to 3 converters
 2024-11-22 13:55:08 +01:00
Stefano Fiorucci
0b2a299378
fix linting of MetaFieldGroupingRanker (#8570) 2024-11-22 13:37:04 +01:00
Bilge Yücel
5cc2df16ee
Update the Studio part on README.md (#8564) 2024-11-21 12:25:35 +03:00
Ulises M
b1353f4f0f
fix: append runtime meta to ChatMessage's extracted meta in AnswerBuilder (#8544) 
* append runtime meta to extracted meta

* add pylint ignore flag to .run()

* explicitly convert reply to string
 2024-11-20 20:07:04 +01:00
Julian Risch
2cc45dd5b9
docs: Update discord invite link in README.md (#8560) 2024-11-20 16:27:31 +01:00
Silvano Cerza
3ef8c081be
fix: OpenAIChatGenerator and OpenAIGenerator crashing when streaming with usage tracking (#8558) 
* Fix OpenAIGenerator crashing with tracking usage with streaming enabled

* Fix OpenAIChatGenerator crashing with tracking usage with streaming enabled

* Add release notes

* Fix linting
 2024-11-20 10:27:22 +01:00
Daria Fokina
3a30ee352e
Update component name in docsrtrings (#8559) 2024-11-19 17:25:15 +01:00
Sebastian Husch Lee
14895f6573
chore: Use token instead of use_auth_token because of deprecation warning (#8552) 
* Use token instead of use_auth_token because of deprecation warning

* Fix test

* pylint

* fix linting

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-11-18 11:58:22 +00:00
Silvano Cerza
bd77120cf3
Fix DocumentSplitter not splitting by function (#8549) 
* Fix DocumentSplitter not splitting by function

* Make the split_by mapping a constant
 2024-11-18 11:54:30 +01:00
Silvano Cerza
cea1e3f07e
Add Zed config folder to .gitignore (#8550) 2024-11-15 17:36:03 +01:00
Ivo Bellin Salarin
c78545dfc0
feat(openai): be tolerant to exceptions (#8526) 
* feat: be tolerant to exceptions

if ever an error is raised by the OpenAI API, don't fail the entire processing

* fix: missing import, string separator

* Enhance error handling

* Use batched from more_itertools for compatibility with older Python versions

* Fix batching and add test

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-11-15 10:52:44 +01:00
Stefano Fiorucci
f085959067
chore: declare requires-python<3.13 in pyproject (#8547) 
* restrict to python<3.13

* try unpinning dulwich

* reintroduce dulwich pin
 2024-11-15 09:28:39 +00:00
Sebastian Husch Lee
e45d3329a1
feat: Adding DALLE image generator (#8448) 
* First pass at adding DALLE image generator

* Add missing header

* Fix tests

* Add tests

* Fix mypy

* Make mypy happy

* More unit tests

* Adding release notes

* Add a test for run

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Fix pylint

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2024-11-14 16:19:49 +01:00
Sriniketh J
a045c0eabb
feat: added split by line to DocumentSplitter (#8525) 
* feat: added split by line to DocumentSplitter

* fix: pr review comments

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2024-11-14 16:09:01 +01:00
Sebastian Husch Lee
0c11c7b98e
fix: Bring in fix from custom nodes (#8539) 
* Bring in fix from custom nodes

* Add to_dict function and test

* reno

* Fix pylint
 2024-11-14 13:00:28 +01:00
Anes Benmerzoug
f5683bc8fa
fix: document joiner division by zero with distribution based rank fusion (#8520) 
* Parametrize document joiner tests with empty lists

* Skip loop in _distribution_based_rank_fusion if document list is empty

* Parametrize test_empty_list with join_mode

* Prevent division by zero in _merge and _reciprocal_rank_fusion

* Add release notes

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-11-14 11:41:28 +00:00
David S. Batista
e5a80722c2
feat: adding metadata grouper component (#8512) 
* initial import

* making tests more readable; adding docstring

* adding release notes

* adding LICENSE header

* Update test/components/rankers/test_metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* refactoring

* fixing docstring

* fixing types

* test docstrings

* renaming test

* handling too-many-arguments

* liting

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* changing name

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* assiging value inside function for re-use

* improving docstring

* updating name to MetaFieldGroupingRanker

* adding to pydocs

* fixing imports

* adding output docstring

* Update haystack/components/rankers/meta_field_grouper_ranker.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/rankers/__init__.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update releasenotes/notes/add-metadata-grouper-21ec05fd4a307425.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update test/components/rankers/test_metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* update docstring tests

* fixing imports

* rename modules for consistency

* fix pydocs

* simplification + more tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2024-11-12 16:01:53 +01:00
David S. Batista
fcdf392bfb
fix: huggingface embedder error messages not being displayed (#8537) 
* initial import

* fixing too-many-arguments - since file is now changed pylint checks it

* more fixes

* disable too-arguments pylint
 2024-11-12 14:59:20 +01:00
David S. Batista
852900d5e3
lowercase comparision (#8532) 2024-11-11 16:33:54 +01:00
Silvano Cerza
ebb45d3d1e
Remove ddtrace version pin (#8529) 2024-11-11 11:21:10 +01:00
Sebastian Husch Lee
911f3523ab
feat: Increase logging transparency for empty Documents during conversion (#8509) 
* Add log lines for PDF conversion and make skipping more explicit in DocumentSplitter

* Add logging statement for PDFMinerToDocument as well

* Add tests

* Remove unused line

* Remove unused line

* add reno

* Add in PDF file

* Update checks in PDF converters and add tests for document splitter

* Revert

* Remove line

* Fix comment

* Make mypy happy

* Make mypy happy
 2024-11-04 09:26:57 +01:00
Bohan Qu
2595e68050
feat: Add TTFT support in OpenAI chat generator (#8444) 
* feat: Add TTFT support in OpenAI generators

* pylint fixes

* correct disable

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
 2024-10-31 16:56:17 +01:00
Sebastian Husch Lee
294a67e426
feat: Adding StringJoiner (#8357) 
* Adding StringJoiner

* Release notes

* Remove typing

* Remove unused import

* Try to fix header

* Fix one test

* Add to docs, move test to behavioral pipeline test

* Undo changes

* Fix test

* Update haystack/components/joiners/string_joiner.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/joiners/string_joiner.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Provide usage example

* Apply suggestions from code review

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-10-30 15:03:41 +00:00
Silvano Cerza
3e30b1db7e Revert afdeaf5a6 and 8271b2d80 2024-10-30 15:41:30 +01:00
Silvano Cerza
8271b2d80b Fix readme_sync.yml unstable version check 2024-10-30 15:34:53 +01:00
Silvano Cerza
afdeaf5a62
Fix readme_sync.yml (#8506) 2024-10-30 15:27:40 +01:00