unstructured

mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-09-07 23:57:32 +00:00

History

feat: use yolox as default to table extraction for pdf/image (#1919 )

- yolox has better recall than yolox_quantized, the current default
model, for table detection
- update logic so that when `infer_table_structure=True` the default
model is `yolox` instead of `yolox_quantized`
- user can still override the default by passing in a `model_name` or
set the env variable `UNSTRUCTURED_HI_RES_MODEL_NAME`

## Test:

Partition the attached file with 

```python
from unstructured.partition.pdf import partition_pdf

yolox_elements = partition_pdf(filename, strategy="hi_re", infer_table_structure=True)
yolox_quantized_elements = partition_pdf(filename, strategy="hi_re", infer_table_structure=True, model_name="yolox_quantized")
```

Compare the table elements between those two and yolox (default)
elements should have more complete table.


[AK_AK-PERS_CAFR_2008_3.pdf](https://github.com/Unstructured-IO/unstructured/files/13191198/AK_AK-PERS_CAFR_2008_3.pdf)

2023-10-27 15:37:45 -05:00

chunking

fix: sectioner does not consider separator length (#1858 )

2023-10-26 21:34:15 +00:00

cleaners

Add clean_ligatures to core cleaners (#1326 )

2023-09-07 21:30:18 +00:00

documents

fix: chunks break on regex-meta changes and regex-meta start/stop not adjusted (#1779 )

2023-10-19 22:16:02 -05:00

embed

feat: Adds local embedding model (#1619 )