2 Commits

Author SHA1 Message Date
Yuming Long
ab5f92dd79
Fix(ingest): Deprecate --s3-url in favor of --remote-url (#616)
* deprecation s3-url

* changelopg and versioin

* download dir not now
2023-05-19 12:11:40 -04:00
cragwolfe
87fd0d01dc
feat: Ingest refactors, doc updates (#243)
- Creates ABC's for ingest connectors
- Updates the s3_connector classes to inherit from ABC's
- Moves s3 test script to it's own file to establish pattern for additional connectors
- Rewrites the Ingest.md doc, including instructions how how to add a connector
- Updates the example s3 ingest script to use the new location for main.py

Note that there were no logic changes, this is essentially a refactoring PR.

Test instructions:

Run ./test_unstructured_ingest/test-ingest.sh and ./examples/ingest/s3-small-batch/ingest.sh.
2023-02-21 10:15:33 -08:00