Christine Straub 237d04c896

feat: improve natural reading order by filtering OCR results (#1768) ...

### Summary
Some `OCR` elements with only spaces in the text have full-page width in
the bounding box, which causes the `xycut` sorting to not work as
expected. Now the logic to parse OCR results removes any elements with
only spaces (more than one space).

---------

Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: christinestraub <christinestraub@users.noreply.github.com>

2023-10-16 23:05:55 +00:00

..

example-docs

roman/cli infer table arg (#1685)

2023-10-12 16:14:53 +00:00

expected-structured-output

feat: improve natural reading order by filtering OCR results (#1768)

2023-10-16 23:05:55 +00:00

failed-partition-docs

roman/ingest continue on error (#1736)

2023-10-12 21:33:10 +00:00

files

Chunking support for SharePoint Connector (#1548)

2023-09-27 21:05:55 +00:00

python

chore: ingest test file cleanup (#1366)

2023-09-21 11:51:08 -07:00

unit

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

.gitignore

fix: update .gitignores to include text comparison files (#1246)

2023-08-30 07:21:04 +00:00

check-diff-expected-output.sh

feat: add file-based access permissions for SharePoint ingest (#1628)

2023-10-13 00:38:08 +00:00

check-num-dirs-output.sh

chore: ingest test file cleanup (#1366)

2023-09-21 11:51:08 -07:00

check-num-files-expected-output.sh

chore: ingest test file cleanup (#1366)

2023-09-21 11:51:08 -07:00

check-num-files-output.sh

chore: ingest test file cleanup (#1366)

2023-09-21 11:51:08 -07:00

check-num-rows-and-columns-output.sh

chore: ingest test file cleanup (#1366)

2023-09-21 11:51:08 -07:00

clean-permissions-files.sh

feat: add file-based access permissions for SharePoint ingest (#1628)

2023-10-13 00:38:08 +00:00

cleanup.sh

chore: don't cleanup ingest test outputs (non-CI) (#1680)

2023-10-07 02:18:37 +00:00

json-to-clean-text-folder.sh

Added plain-text comparison for tests (#1180)

2023-08-29 23:23:14 +00:00

json-to-text.sh

Added plain-text comparison for tests (#1180)

2023-08-29 23:23:14 +00:00

test-ingest-against-api.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-airtable-diff.sh

fix: stop csv and tsv dropping the first line of the file (#1530)

2023-10-16 17:59:35 -05:00

test-ingest-airtable-large.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-azure-cognitive-search.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-azure.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-biomed-api.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-biomed-path.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-box.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-confluence-diff.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-confluence-large.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-delta-table.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-discord.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-dropbox.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-elasticsearch.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-embed.sh

fix: unstructured-ingest embedding KeyError (#1727)

2023-10-12 20:27:30 +00:00

test-ingest-gcs.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-github.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-gitlab.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-google-drive.sh

built(test): exclude version metadata from google drive test (#1682)

2023-10-07 19:34:32 -07:00

test-ingest-jira.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-local-failed-partition.sh

roman/ingest continue on error (#1736)

2023-10-12 21:33:10 +00:00

test-ingest-local-single-file-with-encoding.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-local-single-file-with-pdf-infer-table-structure.sh

roman/cli infer table arg (#1685)

2023-10-12 16:14:53 +00:00

test-ingest-local-single-file.sh

roman/bugfix fix default language ingest option (#1729)

2023-10-12 17:31:23 +00:00

test-ingest-local.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-notion.sh

feat: add Notion connector retry strategy (#1492)

2023-10-10 17:41:18 +00:00

test-ingest-onedrive.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-outlook.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-pdf-fast-reprocess.sh

doc: update comment for ingest test pdf-fast-reprocess (#1733)

2023-10-12 17:33:25 +00:00

test-ingest-s3-compression.sh

roman/fsspec compression support (#1730)

2023-10-16 14:26:30 +00:00

test-ingest-s3-minio.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-s3.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-salesforce.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-sharepoint-embed-cog-index.sh

feat: add file-based access permissions for SharePoint ingest (#1628)

2023-10-13 00:38:08 +00:00

test-ingest-sharepoint.sh

feat: add file-based access permissions for SharePoint ingest (#1628)

2023-10-13 00:38:08 +00:00

test-ingest-slack.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest-wikipedia.sh

refactor: unstructured ingest as a pipeline (#1551)

2023-10-06 18:49:29 +00:00

test-ingest.sh

fix: unstructured-ingest embedding KeyError (#1727)

2023-10-12 20:27:30 +00:00