Yao You d51fb134e6

Feat/improve iou speed (#3582) ...

This PR vectorizes the computation of element overlap to speed up
deduplication process of extracted elements.

## test

This PR adds unit test to the new vectorized IOU and subregion
computation functions.

In addition, running partition on large files with many elements like
this slide:

[002489.pdf](https://github.com/user-attachments/files/16823176/002489.pdf)

shows a reduction of runtime from around 15min on the main branch to
less than 4min with this branch.

Profiling results show that the new implementation greatly reduces the
time cost of computation and now most of the time is spend on getting
the coordinates from a list of bboxes.

![Screenshot 2024-08-30 at 9 29
27 PM](https://github.com/user-attachments/assets/6c186838-54c7-483b-ac3e-7342c23ff3a6)

2024-09-03 00:06:18 +00:00

..

html

feat(chunk): split tables on even row boundaries (#3504)

2024-08-19 18:56:53 +00:00

pdf_image

Feat/improve iou speed (#3582)

2024-09-03 00:06:18 +00:00

utils

feat: partition_pdf() support language specification for PaddleOCR (#3400)

2024-07-16 22:19:25 +00:00

__init__.py

serde tests round-trip through JSON (#1681)

2023-10-12 19:47:55 +00:00

test_api.py

chore: small edits (#3480)

2024-08-06 19:21:43 +00:00

test_auto.py

chore: small edits (#3480)

2024-08-06 19:21:43 +00:00

test_common.py

refactor: restructure PDF/Image example document organization (#3410)

2024-07-18 22:21:32 +00:00

test_constants.py

rfctr(auto): improve expression in tests (#3384)

2024-07-11 19:57:28 +00:00

test_csv.py

feat: Support encoding parameter in partition_csv (#3564)

2024-08-28 14:19:58 +00:00

test_doc.py

rfctr(html): drop convert_and_partition_html() (#3215)

2024-06-17 19:43:18 +00:00

test_docx.py

rfctr(docx): DOCX emits std minified .text_as_html (#3545)

2024-08-21 18:54:21 +00:00

test_email.py

bug: fix file_conversion disk leak (#3562)

2024-08-27 22:02:24 +00:00

test_epub.py

rfctr(html): replace html parser (#3218)

2024-07-11 00:14:28 +00:00

test_json.py

rfctr(file): refactor detect_filetype() (#3429)

2024-07-23 23:18:48 +00:00

test_lang.py

feat: partition_pdf() support language specification for PaddleOCR (#3400)

2024-07-16 22:19:25 +00:00

test_md.py

rfctr: flatten test_unstructured/partition (#3073)

2024-05-23 00:51:08 +00:00

test_msg.py

feat: msg and email metadata (#3444)

2024-08-01 19:24:17 +00:00

test_odt.py

rfctr(html): drop convert_and_partition_html() (#3215)

2024-06-17 19:43:18 +00:00

test_org.py

rfctr(html): drop convert_and_partition_html() (#3215)

2024-06-17 19:43:18 +00:00

test_ppt.py

rfctr: flatten test_unstructured/partition (#3073)

2024-05-23 00:51:08 +00:00

test_pptx.py

fix(pptx): accommodate invalid image/jpg MIME-type (#3475)

2024-08-06 18:48:15 +00:00

test_rst.py

rfctr(html): drop convert_and_partition_html() (#3215)

2024-06-17 19:43:18 +00:00

test_rtf.py

rfctr(html): drop convert_and_partition_html() (#3215)

2024-06-17 19:43:18 +00:00

test_strategies.py

refactor: restructure PDF/Image example document organization (#3410)

2024-07-18 22:21:32 +00:00

test_text_type.py

fix: zero divide in under_non_alpha_ratio (#1753)

2023-10-13 21:20:01 +00:00

test_text.py

Better element IDs - deterministic and document-unique hashes (#2673)

2024-04-24 00:05:20 -07:00

test_tsv.py

rfctr: flatten test_unstructured/partition (#3073)

2024-05-23 00:51:08 +00:00

test_xlsx.py

rfctr: flatten test_unstructured/partition (#3073)

2024-05-23 00:51:08 +00:00

test_xml_partition.py

feat: introduce date_from_file_object parameter to partitions (#2563)

2024-03-18 01:09:44 +00:00