mirror of
https://github.com/Unstructured-IO/unstructured.git
synced 2025-12-27 15:13:35 +00:00
b556d6d575
### Description Migrate over the sharepoint connector to v2 and in the process refactor the majority of the connector. It now pulls in much more content from the SDK on index time, including permissions data is the parameters are passed in. HTML content generated from the SitePage is isolated to the html content in the `CanvasContent1` and `LayoutWebpartsContent` returned by the SDK. Some TODOs were left in there for future iterations. Currently only document and site page content is being pulled in from sharepoint, but sharepoint has more types of content than just that, such as lists. Note left in there to support other sharepoint types. --------- Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: rbiseck3 <rbiseck3@users.noreply.github.com> Co-authored-by: vangheem <vangheem@gmail.com> Co-authored-by: Ahmet Melek <ahmetmeleq@gmail.com> Co-authored-by: Ahmet Melek <39141206+ahmetmeleq@users.noreply.github.com>
90 lines
3.7 KiB
JSON
90 lines
3.7 KiB
JSON
[
|
|
{
|
|
"type": "ListItem",
|
|
"element_id": "fe9e95d69e2fe6e0fcf74f630e24f11f",
|
|
"text": "This is a plain text site page for testing purposes",
|
|
"metadata": {
|
|
"languages": [
|
|
"cat",
|
|
"fra"
|
|
],
|
|
"filename": "This-is-a-title.html",
|
|
"filetype": "text/html",
|
|
"data_source": {
|
|
"url": "https://unstructuredio.sharepoint.com/SitePages/This-is-a-title.aspx",
|
|
"version": "1.0",
|
|
"record_locator": {
|
|
"server_path": "SitePages/This-is-a-title.aspx",
|
|
"site_url": "https://unstructuredio.sharepoint.com"
|
|
},
|
|
"date_modified": "1690787017.0"
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"type": "ListItem",
|
|
"element_id": "bf2f616265a06fa30e74df2cf6291c40",
|
|
"text": "These are bullet points meant for testing",
|
|
"metadata": {
|
|
"languages": [
|
|
"cat",
|
|
"fra"
|
|
],
|
|
"filename": "This-is-a-title.html",
|
|
"filetype": "text/html",
|
|
"data_source": {
|
|
"url": "https://unstructuredio.sharepoint.com/SitePages/This-is-a-title.aspx",
|
|
"version": "1.0",
|
|
"record_locator": {
|
|
"server_path": "SitePages/This-is-a-title.aspx",
|
|
"site_url": "https://unstructuredio.sharepoint.com"
|
|
},
|
|
"date_modified": "1690787017.0"
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"type": "NarrativeText",
|
|
"element_id": "f59e42aff8f1b1ad83f8280a0686eabe",
|
|
"text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales consequat turpis. Etiam vestibulum nisl placerat risus elementum, a sodales purus rhoncus. Sed eget velit pharetra, pretium nisi nec, laoreet ligula. Duis luctus mi in ligula cursus, vel lacinia tortor ultricies. Aenean sit amet sodales odio, a maximus elit. Pellentesque vehicula diam sit amet leo placerat placerat. Integer varius elementum accumsan. Donec posuere elit mauris, eget efficitur nisl viverra vitae.",
|
|
"metadata": {
|
|
"languages": [
|
|
"cat",
|
|
"fra"
|
|
],
|
|
"filename": "This-is-a-title.html",
|
|
"filetype": "text/html",
|
|
"data_source": {
|
|
"url": "https://unstructuredio.sharepoint.com/SitePages/This-is-a-title.aspx",
|
|
"version": "1.0",
|
|
"record_locator": {
|
|
"server_path": "SitePages/This-is-a-title.aspx",
|
|
"site_url": "https://unstructuredio.sharepoint.com"
|
|
},
|
|
"date_modified": "1690787017.0"
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"type": "NarrativeText",
|
|
"element_id": "9fa12141ac0e9ad3d09fe51dc393ad59",
|
|
"text": "Integer at dictum nisi. Cras venenatis non velit in posuere. Curabitur tristique, eros eget tristique pellentesque, neque metus ullamcorper ligula, nec posuere neque lacus nec felis. Nulla a libero eget eros consectetur hendrerit. Pellentesque interdum, diam eget tristique pretium, quam lorem pulvinar lorem, a eleifend nisl lectus at ex. Praesent pulvinar ex ut consequat condimentum. Sed rutrum, erat a hendrerit blandit, urna mauris posuere est, at porttitor risus diam non leo. Nullam rutrum vehicula dolor, quis venenatis ligula rutrum sit amet. Nam massa justo, fermentum in dui lacinia, tincidunt imperdiet nunc. Nam posuere tortor ac lectus elementum, non mollis urna consequat. In interdum non tellus sed pellentesque.",
|
|
"metadata": {
|
|
"languages": [
|
|
"cat",
|
|
"fra"
|
|
],
|
|
"filename": "This-is-a-title.html",
|
|
"filetype": "text/html",
|
|
"data_source": {
|
|
"url": "https://unstructuredio.sharepoint.com/SitePages/This-is-a-title.aspx",
|
|
"version": "1.0",
|
|
"record_locator": {
|
|
"server_path": "SitePages/This-is-a-title.aspx",
|
|
"site_url": "https://unstructuredio.sharepoint.com"
|
|
},
|
|
"date_modified": "1690787017.0"
|
|
}
|
|
}
|
|
}
|
|
] |