Roman Isecke 1df7908f03

feat: save file id for all fsspec connectors if present (#3405) ...

### Description

If the id value exists in the stats response from fsspec, save it as a
`file_id` field in the metadata being persisted on each element.

---------

Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: rbiseck3 <rbiseck3@users.noreply.github.com>

2024-07-19 13:30:21 +00:00

..

dest

refactor: restructure PDF/Image example document organization (#3410)

2024-07-18 22:21:32 +00:00

example-docs

roman/cli infer table arg (#1685)

2023-10-12 16:14:53 +00:00

expected-structured-output

feat: save file id for all fsspec connectors if present (#3405)

2024-07-19 13:30:21 +00:00

failed-partition-docs

roman/ingest continue on error (#1736)

2023-10-12 21:33:10 +00:00

files

rfctr: Implement Azure Cognitive Search V2 Destination Connector (#3311)

2024-07-09 12:19:15 +00:00

metrics

BREAKING CHANGE: revert table extraction off by default for PDFs and images (#3035)

2024-05-17 15:28:11 +00:00

python

refactor: implement databricks volumes v2 dest connector (#3334)

2024-07-03 19:01:16 +00:00

src

refactor: restructure PDF/Image example document organization (#3410)

2024-07-18 22:21:32 +00:00

unit

refactor: restructure PDF/Image example document organization (#3410)

2024-07-18 22:21:32 +00:00

.gitignore

ci: reorganize eval output folders and add azure to matrix test (#2093)

2023-11-21 20:04:30 +00:00

check-diff-evaluation-metrics.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

check-diff-expected-output.sh

Fix: ingest test fixtures update pr (#2881)

2024-04-15 17:47:22 +00:00

check-num-dirs-output.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

check-num-files-expected-output.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

check-num-files-output.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

check-num-rows-and-columns-output.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

clean-permissions-files.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

cleanup.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

evaluation-ingest-cp.sh

chore: shell scripts default indent of 2 instead of 4 (#2287)

2023-12-19 07:48:21 +00:00

evaluation-metrics.sh

feat(eval): table evaluation metrics (#2558)

2024-02-22 16:35:46 +00:00

json-to-clean-text-folder.sh

chore: add shfmt (#2246)

2023-12-12 01:04:15 +00:00

json-to-text.sh

chore: add shfmt (#2246)

2023-12-12 01:04:15 +00:00

test-help.sh

Roman/bugfix conflicting event loop ingest (#3264)

2024-06-24 18:47:37 +00:00

test-ingest-dest.sh

refactor: implement databricks volumes v2 dest connector (#3334)

2024-07-03 19:01:16 +00:00

test-ingest-src.sh

ci: disable sharepoint ingest test (#3393)

2024-07-14 01:58:08 +00:00