Christine Straub be88eef06f

perf: optimize pdfminer image cleanup process for improved performance (#3630) ...

This PR enhances `pdfminer` image cleanup process by repositioning the
duplicate image removal step. It optimizes the removal of duplicated
pdfminer images by performing the cleanup before merging elements,
rather than after. This improvement reduces execution time and enhances
the overall processing speed of PDF documents.

---------

Co-authored-by: Yao You <theyaoyou@gmail.com>

2024-09-19 14:05:05 +00:00

..

chunking

feat(chunk): split tables on even row boundaries (#3504)

2024-08-19 18:56:53 +00:00

cleaners

rfctr: prepare to add orig_elements serde (#2668)

2024-03-20 21:27:59 +00:00

common

feat(chunk): split tables on even row boundaries (#3504)

2024-08-19 18:56:53 +00:00

documents

rfctr(html): replace html parser (#3218)

2024-07-11 00:14:28 +00:00

embed

Potter/mixedbread embedder (#3513)

2024-08-27 14:52:13 +00:00

file_utils

chore(file): remove dead code (#3645)

2024-09-19 06:45:33 +00:00

ingest/utils

enhancement: apply tar filters when using python 3.12 or above (#3124)

2024-06-05 18:28:59 +00:00

metrics

rfctr(file): refactor detect_filetype() (#3429)

2024-07-23 23:18:48 +00:00

nlp

fix(CVE-2024-39705): update to latest nltk version (#3512)

2024-08-13 09:39:29 -04:00

partition

perf: optimize pdfminer image cleanup process for improved performance (#3630)

2024-09-19 14:05:05 +00:00

staging

bug: fix file_conversion disk leak (#3562)

2024-08-27 22:02:24 +00:00

vcr_fixtures/cassettes

chore: Re-enable test_upload_label_studio_data_with_sdk (#674)

2023-06-02 23:38:43 +00:00

__init__.py

chore: Reorganize partition bricks under partition directory (#76)

2022-11-21 22:27:23 +00:00

test_utils.py

rfctr(chunk): prep for adding TableSplitter (#3510)

2024-08-12 18:04:49 +00:00

unit_utils.py

bug: fix file_conversion disk leak (#3562)

2024-08-27 22:02:24 +00:00