unstructured/test_unstructured_ingest/test-ingest-against-api.sh
Matt Robinson 894a190001
enhancement: check for copy protection on PDFs and fallback to hi res when necessary (#514)
* function to check if pdf is extractable

* add fallback logic for unextractable pdfs

* tests for docs with copy protection

* add test for unprocessable pdf

* update docs

* changelog and version

* update logic for images; reset file before proceeding

* 3 files for api tests

* docs update
2023-04-21 21:35:43 +00:00

23 lines
505 B
Bash
Executable File

#!/usr/bin/env bash
set -e
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)
cd "$SCRIPT_DIR"/.. || exit 1
PYTHONPATH=. ./unstructured/ingest/main.py \
--local-input-path example-docs \
--local-file-glob "*.pdf" \
--structured-output-dir api-ingest-output \
--partition-by-api \
--verbose \
--reprocess
set +e
if [ "$(find 'api-ingest-output' -type f -printf '.' | wc -c)" != 3 ]; then
echo
echo "3 files should have been created."
exit 1
fi