Steve Canny 208c7edc52

rfctr(csv): minify HTML and table text is cct (#3733) ...

**Summary**
Eliminate historical "idiosyncracies" of `table.metadata.text_as_html`
HTML introduced by `partition_csv()`. Produce minified `.text_as_html`
consistent with that formed by chunking.

**Additional Context**
- CSV `.metadata.text_as_html` is minified (no extra whitespace or
thead, tbody, tfoot elements).
- `table.text` is clean-concatenated-text (CCT) of table.

---------

Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: scanny <scanny@users.noreply.github.com>

2024-10-19 06:49:09 +00:00

..

airtable-diff

Better element IDs - deterministic and document-unique hashes (#2673)

2024-04-24 00:05:20 -07:00

astradb/ingest_test_src

chore: rename astra to astradb (#3458)

2024-08-05 20:41:02 +00:00

azure

feat/remove ingest code, use new dep for tests (#3595)

2024-10-15 10:01:34 -05:00

biomed-api

feat: improve pdfminer element processing (#3618)

2024-09-12 21:17:27 +00:00

biomed-path/07/07

feat: improve pdfminer element processing (#3618)

2024-09-12 21:17:27 +00:00

box

feat/remove ingest code, use new dep for tests (#3595)

2024-10-15 10:01:34 -05:00

confluence-diff

feat(chunk): split tables on even row boundaries (#3504)

2024-08-19 18:56:53 +00:00

delta-table

rfctr(csv): minify HTML and table text is cct (#3733)

2024-10-19 06:49:09 +00:00

discord

Better element IDs - deterministic and document-unique hashes (#2673)

2024-04-24 00:05:20 -07:00

dropbox

feat/remove ingest code, use new dep for tests (#3595)

2024-10-15 10:01:34 -05:00

elasticsearch

rfctr(part): remove double-decoration 4 (#3690)

2024-10-03 16:41:31 +00:00

embed

fix: revert dropping of filename extension for some connectors (#3109)

2024-05-29 19:14:22 +00:00

embed-bedrock

rfctr(part): remove double-decoration 4 (#3690)

2024-10-03 16:41:31 +00:00

embed-mixedbreadai

Potter/mixedbread embedder (#3513)

2024-08-27 14:52:13 +00:00

embed-vertexai

fix: revert dropping of filename extension for some connectors (#3109)

2024-05-29 19:14:22 +00:00

embed-voyageai

feat: add VoyageAI embeddings (#3069) (#3099)

2024-05-24 21:48:35 +00:00

gcs

fix(xlsx): XLSX emits std minified .text_as_html (#3558)

2024-10-17 22:05:11 +00:00

github

rfctr(html): replace html parser (#3218)

2024-07-11 00:14:28 +00:00

google-drive

feat: improve pdfminer element processing (#3618)

2024-09-12 21:17:27 +00:00

hubspot

Better element IDs - deterministic and document-unique hashes (#2673)

2024-04-24 00:05:20 -07:00

jira-diff

Better element IDs - deterministic and document-unique hashes (#2673)

2024-04-24 00:05:20 -07:00

kafka

feat: Kafka source and destination connector (#3176)

2024-06-22 23:26:23 +00:00

local-single-file

rfctr(part): remove double-decoration 4 (#3690)

2024-10-03 16:41:31 +00:00

local-single-file-basic-chunking

Roman/fix ingest async connectors (#3210)

2024-06-17 16:55:19 +00:00

local-single-file-chunk-no-orig-elements

refactor: restructure PDF/Image example document organization (#3410)

2024-07-18 22:21:32 +00:00

local-single-file-with-encoding

rfctr(html): replace html parser (#3218)

2024-07-11 00:14:28 +00:00

local-single-file-with-pdf-infer-table-structure

feat: improve pdfminer element processing (#3618)

2024-09-12 21:17:27 +00:00

mongodb

rfctr(csv): minify HTML and table text is cct (#3733)

2024-10-19 06:49:09 +00:00

notion

rfctr(csv): minify HTML and table text is cct (#3733)

2024-10-19 06:49:09 +00:00

onedrive/utic-test-ingest-fixtures

fix(xlsx): XLSX emits std minified .text_as_html (#3558)

2024-10-17 22:05:11 +00:00

opensearch

rfctr(part): remove double-decoration 4 (#3690)

2024-10-03 16:41:31 +00:00

outlook

rfctr(email): eml partitioner rewrite (#3694)

2024-10-16 02:02:33 +00:00

pdf-fast-reprocess

fix: revert dropping of filename extension for some connectors (#3109)

2024-05-29 19:14:22 +00:00

s3

feat: improve pdfminer element processing (#3618)

2024-09-12 21:17:27 +00:00

s3-minio

rfctr(csv): minify HTML and table text is cct (#3733)

2024-10-19 06:49:09 +00:00

salesforce

rfctr(email): eml partitioner rewrite (#3694)

2024-10-16 02:02:33 +00:00

sftp/folder1

rfctr(csv): minify HTML and table text is cct (#3733)

2024-10-19 06:49:09 +00:00

Sharepoint

rfctr: Implement Sharepoint V2 Source Connector (#3314)

2024-07-09 09:52:59 +00:00

Sharepoint-with-permissions

rfctr: Implement Sharepoint V2 Source Connector (#3314)

2024-07-09 09:52:59 +00:00

slack

fix: update slack test to point to new channel (#3328)

2024-07-01 18:11:21 +00:00