unstructured

mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-12-12 15:42:19 +00:00

History

fix languages 500 error with empty string for ocr_languages (#1968 )

Closes #1870 
Defining both `languages` and `ocr_languages` raises a ValueError, but
the api defaults to `ocr_languages` being an empty string, so if users
define `languages` they are automatically hitting the ValueError.

This fix checks if `ocr_languages` is an empty string and converts it to
`None` to avoid this.

### Testing
On the main branch, the following will raise the ValueError, but it will
correctly partition on this branch
```
from unstructured.partition.auto import partition
filename = "example-docs/category-level.docx"
elements = partition(filename,languages=['spa'],ocr_languages="")

elements[0].metadata.languages
```

---------

Co-authored-by: yuming <305248291@qq.com>
Co-authored-by: Yuming Long <63475068+yuming-long@users.noreply.github.com>
Co-authored-by: Austin Walker <awalk89@gmail.com>

2023-11-01 22:02:00 +00:00

chunking

feat: clean pdfminer elements inside tables (#1808 )

2023-10-30 07:10:51 +00:00

cleaners

Add clean_ligatures to core cleaners (#1326 )

2023-09-07 21:30:18 +00:00

documents

fix: chunks break on regex-meta changes and regex-meta start/stop not adjusted (#1779 )

2023-10-19 22:16:02 -05:00

embed

feat: Adds local embedding model (#1619 )