mirror of
https://github.com/Unstructured-IO/unstructured.git
synced 2025-09-25 08:19:22 +00:00
db766402a4
This PR resolves [CORE-2453](https://unstructured-ai.atlassian.net/browse/CORE-2453): - parametrizes the output folder so that ingest output files can be saved other than the same place where the scripts are; this is set by env `OUTPUT_ROOT` - parametrize the python path `PYTHONPATH` to first check existing definition before default to `.`, the current folder - parametrize the run script that carries out ingest using `RUN_SCRIPT`, default is still `./unstructured/ingest/main.py` These changes allows us to run ingest test with more control. To test: - run `OUTPUT_ROOT=/tmp ./test_unstructured_ingest/src/local-single-file.sh`: the output now should be in `/tmp` instead of in the ingest test folder - run `RUN_SCRIPT=/hope/you/do/not/have/this/folder ./test_unstructured_ingest/src/local-single-file.sh` would raise an error because system can't find `/hope/you/do/not/have/this/folder` - run `RUN_SCRIPT=./unstructured/ingest/main.py ./test_unstructured_ingest/src/local-single-file.sh` should run as normal - do the following ```bash cp ./unstructured/ingest/main.py /tmp/main.py OUTPUT_ROOT=/tmp PYTHONPATH=$(pwd) RUN_SCRIPT=./unstructured/ingest/main.py ./test_unstructured_ingest/src/local-single-file.sh ``` This will run and generate output at `/tmp` [CORE-2453]: https://unstructured-ai.atlassian.net/browse/CORE-2453?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ