yujunjun/unstructured
1
0
Fork 0
mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-07-23 09:00:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
unstructured/test_unstructured_ingest/expected-structured-output/confluence-diff/MFS/1605928.json
Michał Martyniak 2d1923ac7e
Better element IDs - deterministic and document-unique hashes (#2673) 
Part two of: https://github.com/Unstructured-IO/unstructured/pull/2842

Main changes compared to part one:
* hash computation includes element's sequence number on page, page
number, document filename and its text
* there are more test for deterministic behavior of IDs returned by
partitioning functions + their uniqueness (guaranteed at the document
level, and high probability across multiple documents)

This PR addresses the following issue:
https://github.com/Unstructured-IO/unstructured/issues/2461
2024-04-24 00:05:20 -07:00

245 lines
7.0 KiB
JSON
Raw Permalink Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

[
{
"element_id": "fff28f1810e9e7fadc4cee805265466e",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "\\uD83D\\uDDD3 Date",
"type": "Title"
},
{
"element_id": "6b5975666ff11ded81460bd163889869",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "\\uD83D\\uDC65 Participants",
"type": "Title"
},
{
"element_id": "efdb80f487530df5dedbd40d0ccaa885",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "",
"type": "ListItem"
},
{
"element_id": "4ac682eab89d8495a7cd07466757029f",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "",
"type": "ListItem"
},
{
"element_id": "5fa89c297204b53023cb3e19ce4385f9",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "\\uD83E\\uDD45 Goals",
"type": "Title"
},
{
"element_id": "c8ef6f62fd82a1be88733eddb3fa7c75",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "",
"type": "ListItem"
},
{
"element_id": "3cb7984bcb4433a53358aa980beef286",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "\\uD83D\\uDDE3 Discussion topics",
"type": "Title"
},
{
"element_id": "c114a47ef12aa1eaa55603c3f4dd87e0",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1,
"text_as_html": "<table><tr><td>Time</td><td>Item</td><td>Presenter</td><td>Notes</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td></tr></table>"
},
"text": "Time Item Presenter Notes",
"type": "Table"
},
{
"element_id": "2473ced77e8610dbbdb2e1a09e36e376",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "✅ Action items",
"type": "Title"
},
{
"element_id": "d8f05203bd43f4b13777aca609718e3d",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "",
"type": "ListItem"
},
{
"element_id": "81400b25b68a00ad60090b2b5e4bd744",
"metadata": {
"data_source": {
"date_created": "2023-07-09T12:54:45.162000",
"date_modified": "2023-07-09T12:54:45.162000",
"record_locator": {
"page_id": "1605928",
"url": "https://unstructured-ingest-test.atlassian.net"
},
"url": "https://unstructured-ingest-test.atlassian.net/wiki/rest/api/content/1605928",
"version": "1"
},
"filetype": "text/html",
"languages": [
"eng"
],
"page_number": 1
},
"text": "⤴ Decisions",
"type": "Title"
}
]
Reference in New Issue View Git Blame Copy Permalink