yujunjun/unstructured
1
0
Fork 0
mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-07-09 10:05:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
unstructured/docs/source/ingest/source_connectors.rst

38 lines
1.1 KiB
ReStructuredText
Raw Permalink Normal View History

roman/delta lake dest connector (#1385) ### Description Add delta table downstream destination connector Closes https://github.com/Unstructured-IO/unstructured/issues/1415
2023-09-15 18:13:39 -04:00
Source Connectors
=================
Connect to your favorite data storage platforms for effortless batch processing of your files.
feat: Enable remote chunking via unstructured-ingest (#2905) Update: The cli shell script works when sending documents to the free api, but the paid api is down, so waiting to test against it. - The first commit adds docstrings and fixes type hints. - The second commit reorganizes `test_unstructured_ingest` so it matches the structure of `unstructured/ingest`. - The third commit contains the primary changes for this PR. - The `.chunk()` method responsible for sending elements to the correct method is moved from `ChunkingConfig` to `Chunker` so that `ChunkingConfig` acts as a config object instead of containing implementation logic. `Chunker.chunk()` also now takes a json file instead of a list of elements. This is done to avoid redundant serialization if the file is to be sent to the api for chunking. --------- Co-authored-by: Ahmet Melek <39141206+ahmetmeleq@users.noreply.github.com>
2024-04-24 19:24:58 -05:00
We are constantly adding new data connectors and if you don't see your favorite platform let us know
fix: update slack link to link shortener (#2010) Per @tabossert we're now using a link shortener behind which we can rotate the link to keep it current. That way we (🤞 ) never have to update this here again. #### Testing: Links should work. No more links should exist in the documentation except this one.
2023-11-06 09:47:18 -06:00
in our community `Slack. <https://short.unstructured.io/pzw05l7>`_
roman/delta lake dest connector (#1385) ### Description Add delta table downstream destination connector Closes https://github.com/Unstructured-IO/unstructured/issues/1415
2023-09-15 18:13:39 -04:00
.. toctree::
:maxdepth: 1
source_connectors/airtable
source_connectors/azure
source_connectors/biomed
source_connectors/box
source_connectors/confluence
source_connectors/delta_table
source_connectors/discord
source_connectors/dropbox
source_connectors/elasticsearch
source_connectors/github
source_connectors/gitlab
source_connectors/google_cloud_storage
source_connectors/google_drive
source_connectors/jira
source_connectors/local_connector
feat: add MongoDB source connector (#2393) Adds MongoDB as a source (we already had it as a destination connector) --------- Co-authored-by: potter-potter <david.potter@gmail.com>
2024-01-16 12:56:29 -08:00
source_connectors/mongodb
roman/delta lake dest connector (#1385) ### Description Add delta table downstream destination connector Closes https://github.com/Unstructured-IO/unstructured/issues/1415
2023-09-15 18:13:39 -04:00
source_connectors/notion
source_connectors/onedrive
Kapa.ai widget installation (#2418) To test: > cd docs && make html > click "Ask AI" button on the bottom right-hand corner Changelogs: * Installed kapa.ai widget * fixed sphinx errors in opensearch & elasticsearch documentation
2024-01-17 16:17:11 -08:00
source_connectors/opensearch
roman/delta lake dest connector (#1385) ### Description Add delta table downstream destination connector Closes https://github.com/Unstructured-IO/unstructured/issues/1415
2023-09-15 18:13:39 -04:00
source_connectors/outlook
source_connectors/reddit
source_connectors/s3
source_connectors/salesforce
feat: Add sftp source connector (#2163) Adds source connector for SFTP which uses fsspec and paramiko via fsspec. Paramiko is the standard sftp package for python used in pysftp etc... ``` --username foo \ --password bar \ --remote-url sftp://localhost:47474/upload/ ``` Will only download a specifically requested file if it has an extension. (i.e. `--remote-url sftp://localhost:47474/upload/bob.zip`) It will treat any other remote_url as a folder path. This is intentional. --------- Co-authored-by: potter-potter <david.potter@gmail.com>
2023-12-07 11:33:19 -08:00
source_connectors/sftp
roman/delta lake dest connector (#1385) ### Description Add delta table downstream destination connector Closes https://github.com/Unstructured-IO/unstructured/issues/1415
2023-09-15 18:13:39 -04:00
source_connectors/sharepoint
source_connectors/slack
source_connectors/wikipedia
Reference in New Issue Copy Permalink