mirror of
https://github.com/Unstructured-IO/unstructured.git
synced 2025-08-15 12:19:36 +00:00
d5fd21f0fd
Closes #1064 When using the `--partition-by-api` flag via unstructured-ingest, none of the partition arguments are forwarded, meaning that these options are disregarded. With this change, we now pass through all of the relevant partition arguments to the api. ## Changes * parse and pass relevant partition arguments to the api in unstructured-ingest * bonus: leverage an existing `partition.api` function to call out to the api rather than including duplicative request logic in unstructured ingest * bonus: --pdf-infer-table-structure is now a flag not an arg (it defaults false anyways, this is more succinct and consistent with similar parameters) * bonus: adds `hi_res_model_name` so a user can specify the model to leverage when using a hi_res strategy. ## Testing * update against_api.sh source test script to specify a partition argument and validates that the response from the api respected the argument * manually ran a request and validated that it was processed with chipper as specified (not sure if we want to bake a chipper request into the ci tests) (validated that the response leveraged the chipper model): ``` PYTHONPATH=. ./unstructured/ingest/main.py \ local \ --output-dir /tmp/ingest-requests/chipper \ --verbose \ --reprocess \ --strategy hi_res \ --partition-by-api \ --hi-res-model-name chipper \ --api-key "$API_KEY" \ --input-path 'example-docs/layout-parser-paper-with-table.pdf' ```