3 Commits

Author SHA1 Message Date
cragwolfe
abe4e8191a
chore: ingest-script cleanup, better skip condition (#2094)
When testing ingest tests, one often wants to keep the .json output or
generated metrics files around for inspection after the fact. This
updates the bash condition to actually honor the comment that mentions

    # export UNSTRUCTURED_CLEANUP_DEV_FIXTURES=1
    
** Test Instructions **

Run:

    export UNSTRUCTURED_CLEANUP_DEV_FIXTURES=1
    ./test_unstructured_ingest/src/s3.sh
    ./test_unstructured_ingest/evaluation-metrics.sh text-extraction
    
and witness test directories/files do not get cleaned up. E.g.,
`test_unstructured_ingest/metrics-tmp/`. One can also add a `set -x` at
the top of test_unstructured_ingest/cleanup.sh to see what is getting
skipped (it's a lot!).
2023-11-15 22:28:04 -08:00
cragwolfe
46cb1b642a
chore: don't cleanup ingest test outputs (non-CI) (#1680)
When running test-ingest test fixtures locally (but not in CI), keep
output .json's and other workdir artifacts around for the convenience of
debugging.

**Test Instructions**

Run 

    bash -x ./test_unstructured_ingest/test-ingest-azure.sh

and witness output .json's are visible. Yay! Now, to instead clean up
output .json's and workdir, run:

UNSTRUCTURED_CLEANUP_DEV_FIXTURES=1 bash -x
./test_unstructured_ingest/test-ingest-azure.sh
    
and witness the files have been cleaned up. Yay!
2023-10-07 02:18:37 +00:00
Roman Isecke
e88f7d9eab
chore: ingest test file cleanup (#1366) 2023-09-21 11:51:08 -07:00