7 Commits

Author SHA1 Message Date
Roman Isecke
22e568cf64
roman/bugfix fix default language ingest option (#1729)
### Description
Set language to None by default. Update ingest test to use local file
used in language unit tests to validate.

---------

Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: rbiseck3 <rbiseck3@users.noreply.github.com>
2023-10-12 17:31:23 +00:00
John
9500d04791
detect document language across all partitioners (#1627)
### Summary
Closes #1534 and #1535
Detects document language using `langdetect` package. 
Creates new kwargs for user to set the document language (`languages`)
or detect the language at the element level instead of the default
document level (`detect_language_per_element`)

---------

Co-authored-by: shreyanid <42684285+shreyanid@users.noreply.github.com>
Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: Coniferish <Coniferish@users.noreply.github.com>
Co-authored-by: cragwolfe <crag@unstructured.io>
Co-authored-by: Austin Walker <austin@unstructured.io>
2023-10-11 01:47:56 +00:00
rvztz
d8a36af08c
fix: Sharepoint connector server_path issue (#1497) 2023-09-26 14:25:35 -07:00
ryannikolaidis
955efac935
fix: SharePoint connector fails if any document has an unsupported filetype (#1493) 2023-09-22 18:47:28 +00:00
rvztz
424852ab39
feat: adds data source properties to Sharepoint and Outlook (#1278) 2023-09-20 09:13:35 +00:00
ryannikolaidis
076b1e38f4
feat: serialize ingest docs as json (#1178) 2023-08-31 01:48:41 +00:00
rvztz
dee9b405cd
feat: Sharepoint connector (#918) 2023-08-10 09:37:58 -07:00