John 2f553333bd

refactor text.py (#1872) ...

### Summary
Closes #1520 
Partial solution to #1521 

- Adds an abstraction layer between the user API and the partitioner
implementation
- Adds comments explaining paragraph chunking
- Makes edits to pass strict type-checking for both text.py and
test_text.py

2023-11-01 17:44:55 -05:00

..

chunking

feat: clean pdfminer elements inside tables (#1808)

2023-10-30 07:10:51 +00:00

cleaners

Add clean_ligatures to core cleaners (#1326)

2023-09-07 21:30:18 +00:00

documents

fix: chunks break on regex-meta changes and regex-meta start/stop not adjusted (#1779)

2023-10-19 22:16:02 -05:00

embed

feat: Adds local embedding model (#1619)

2023-10-19 11:51:36 -05:00

file_utils

Chore: Libmagic detection for "application/octet-stream" when it is not a zip file. (#1347)

2023-09-08 18:49:00 +00:00

metrics

fix: avoid loop through None (#1975)

2023-11-01 20:50:34 +00:00

nlp

Add check for duplicate click options (#1775)

2023-10-20 14:00:19 +00:00

partition

refactor text.py (#1872)

2023-11-01 17:44:55 -05:00

staging

feat: add filter element types as post processing function (#1014)

2023-08-03 10:50:35 -04:00

vcr_fixtures/cassettes

chore: Re-enable test_upload_label_studio_data_with_sdk (#674)

2023-06-02 23:38:43 +00:00

__init__.py

chore: Reorganize partition bricks under partition directory (#76)

2022-11-21 22:27:23 +00:00

test_utils.py

feat: method to catch and classify overlapping bounding boxes (#1803)

2023-10-25 12:17:34 +00:00

unit_utils.py

serde tests round-trip through JSON (#1681)

2023-10-12 19:47:55 +00:00