Steve Canny 1af41d5f90

feat(chunking): add .orig_elements behavior to chunking (#2656) ...

**Summary**
Add the actual behavior to populate `.metadata.orig_elements` during
chunking, when so instructed by the `include_orig_elements` option.

**Additional Context**
The underlying structures to support this, namely the
`.metadata.orig_elements` field and the `include_orig_elements` chunking
option, were added in closely prior PRs. This PR adds the behavior to
actually populate that metadata field during chunking when the option is
set.

2024-03-18 19:27:39 +00:00

..

chunking

feat(chunking): add .orig_elements behavior to chunking (#2656)

2024-03-18 19:27:39 +00:00

cleaners

Add clean_ligatures to core cleaners (#1326)

2023-09-07 21:30:18 +00:00

documents

fix(html): unequal row lengths in HTMLTable.text_as_html (#2345)

2024-01-04 21:53:19 +00:00

embed

Renamed OpenAiEmbeddingConfig dataclass (#2546)

2024-02-14 17:24:52 +00:00

file_utils

fix: don't treat double quote enclosed text as JSON (#2544)

2024-02-14 13:41:43 +00:00

metrics

feat: grouping eval takes list of filenames (#2635)

2024-03-17 17:19:55 +00:00

nlp

Add check for duplicate click options (#1775)

2023-10-20 14:00:19 +00:00

partition

feat: introduce date_from_file_object parameter to partitions (#2563)

2024-03-18 01:09:44 +00:00

staging

fix: remove none value keys from flattened dictionary (#2442)

2024-01-23 21:52:11 +00:00

vcr_fixtures/cassettes

chore: Re-enable test_upload_label_studio_data_with_sdk (#674)

2023-06-02 23:38:43 +00:00

__init__.py

chore: Reorganize partition bricks under partition directory (#76)

2022-11-21 22:27:23 +00:00

test_utils.py

chore: add tests and small fixes in utils.py (#2554)

2024-03-06 21:58:10 +00:00

unit_utils.py

rfctr: prepare for adding metadata.orig_elements field (#2647)

2024-03-14 21:31:58 +00:00