* Fixes to setting StreamingChunk.index properly and refactoring tests for conversion
* Make _convert_chat_completion_chunk_to_streaming_chunk a member of OpenAIChatGenerator so we can overwrite it in integrations that inherit from it
* Fixes
* Modify streaming chunk to accept a list of tool call deltas.
* Fix tests
* Fix mypy and update original reno
* Undo change
* Update conversion to return a single streaming chunk
* update to print streaming chunk
* Fix types
* PR comments
* Start expanding StreamingChunk
* First pass at expanding Streaming Chunk
* Working version!
* Some tweaks and also make ToolInvoker stream a chunk with a finish reason
* Properly update test
* Change to tool_name, remove kw_only since its python 3.10 only and update HuggingFaceAPIChatGenerator to start following new StreamingChunk
* Add reno
* Some cleanup
* Fix unit tests
* Fix mypy and integration test
* Fix pylint
* Start refactoring huggingface local api
* Refactor openai generator and chat generator to reuse util methods
* Did some reorg
* Reusue utility method in HuggingFaceAPI
* Get rid of unneeded default values in tests
* Update conversion of streaming chunks to chat message to not rely on openai dataclass anymore
* Fix tests and loosen check in StreamingChunk post_init
* Fixes
* Fix license header
* Add start and index to HFAPIGenerator
* Fix mypy
* Clean up
* Update haystack/components/generators/utils.py
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
* Update haystack/components/generators/utils.py
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
* Change StreamingChunk.start to only a bool
* PR comments
* Fix unit test
* PR comment
* Fix test
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
* Refactor HFAPI Chat Generator
* Add component info to generators
* Fix type hint
* Add reno
* Fix unit tests
* Remove incorrect dev comment
* Move _convert_streaming_chunks_to_chat_message to utils file
* Starting property schema refactor
* Adding more tests
* More tests
* Handle null type explicitly
* More updates of tests to accomodate Optional properly
* Fix more tests
* Remove unecessary check
* Some cleanup
* Update test
* Add reno
* Fix typing
* Add license header
* Use docstrings of dataclasses in parameter spec generation
* More tests of Haystack dataclass types
* Properly handle Sequence
* Fix license header
* Update OpenAI tests to add more complicated tool parameter signature
* Properly set required for dataclasses
* Add integration test for azure that includes additionalProperties
* Add more complicated integration test for HuggingFaceAPIChatGenerator
* Alternate approach using pydantic like we do in from_function.py
* Cleanup and fix other affected tests
* Fix mypy
* PR comments
* PR comment
* Remove test from HF API
* Update reno
* Update reno
* feat: Add sanitization for Meta field during serialization
* Revert "feat: Add sanitization for Meta field during serialization"
This reverts commit c529f7c25b69aed626bb2072c8bf171815b591cc.
* feat: add nested serialization in openai usage object
* add reno
* add nested serialization in OpenAiChatGenerator
* Update releasenotes/notes/nested-serialization-openai-usage-object-3817b07342999edf.yaml
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
* merge tests
* Adjust the test
---------
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
* AzureOCR: convert integration test to unit test and simplify
* clean up HuggingFaceAPITextEmbedder
* clean up LinkContentFetcher
* simplify HuggingFaceLocalGenerator
* clean up OpenAIGenerator
* OpenAIChatGenerator
* SentenceTransformersDiversityRanker
* TransformersSimilarityRanker
* ChatMessage: rm outdated tests
* fail fast false
* typo
* Fix for handling wrapped ChatCompletion instances in streaming (used by tools like weave)
* Add release note
* Applied same fix to OpenAIGenerator ; Refactoring ; Update release note
* Fix integration test error after refactoring
* fix: manage max_retries=0 in AzureOpenAIGenerator and AzureOpenAIChatGenerator
* fix: manage max_retries=0 in AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder
* update Tool,ToolInvoker,ComponentTool,create_tool_from_function
* add State and its utils
* add tests for State and its utils
* update tests for Tool etc.
* reno
* fix circular imports
* update experimental imports in tests
* fix unit tests
* fix ChatGenerator unit tests
* mypy
* add State to init and pydoc
* explain State in more detail in release note
* add test from #8913
* re-add _check_duplicate_tool_names and refactor imports
* rename inputs and outputs
* Start adding support for passing callable to Azure components
* Add to chat version
* Fix test
* Add reno
* Add support to azure doc and text embedder
* Rename
* update llm metadata extractor
* Add tests for text embedder
* Update tests
* Remove unused fixture and import
* Update reno
* fix: set ComponentTool addtionalProperties for OpenAI tools_strict=True
* add reno
* Move the additionalProperties into the OpenAIChatGenerator
* Remove
* Put additionalProperties into the correct place
* Fix test
* Update releasenotes/notes/fix-componenttool-for-openai-tools_strict-998e5cd7ebc6ec19.yaml
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
---------
Co-authored-by: Sebastian Husch Lee <sebastian.lee@deepset.ai>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Starting to refactor type util tests to be more systematic
* refactoring
* Expand tests
* Update to type utils
* Add missing subclass check
* Expand and refactor tests, introduce type_validation Literal
* More test refactoring
* Test refactoring, adding type validation variable to pipeline base
* Update relaxed version of type checking to pass all newly added tests
* trim whitespace
* Add tests
* cleanup
* Updates docstrings
* Add reno
* docs
* Fix mypy and add docstrings
* Changes based on advice from Tobi
* Remove unused imports
* Doc strings
* Add connection type validation to to_dict and from_dict
* Update tests
* Fix test
* Also save connection_type_validation at global pipeline level
* Fix tests
* Remove connection type validation from the connect level, only keep at pipeline level
* Formatting
* Fix tests
* formatting
* Revert "test: skip HF API live integration tests (#8889)"
This reverts commit 56a3a9bd61b7391ae91e3d8179b3b33918ef4932.
* Replace zephyr-7b-beta model with SmolLM2-1.7B-Instruct
* Use zephyr-7b-beta model but extend instructions
---------
Co-authored-by: David S. Batista <dsbatista@gmail.com>
