### Description
Given all the shell files that now exist in the repo, would be nice to
have linting/formatting around them (in addition to the existing
shellcheck which doesn't do anything to format the shell code). This PR
introduces `shfmt` to both check for changes and apply formatting when
the associated make targets are called.
When testing ingest tests, one often wants to keep the .json output or
generated metrics files around for inspection after the fact. This
updates the bash condition to actually honor the comment that mentions
# export UNSTRUCTURED_CLEANUP_DEV_FIXTURES=1
** Test Instructions **
Run:
export UNSTRUCTURED_CLEANUP_DEV_FIXTURES=1
./test_unstructured_ingest/src/s3.sh
./test_unstructured_ingest/evaluation-metrics.sh text-extraction
and witness test directories/files do not get cleaned up. E.g.,
`test_unstructured_ingest/metrics-tmp/`. One can also add a `set -x` at
the top of test_unstructured_ingest/cleanup.sh to see what is getting
skipped (it's a lot!).
When running test-ingest test fixtures locally (but not in CI), keep
output .json's and other workdir artifacts around for the convenience of
debugging.
**Test Instructions**
Run
bash -x ./test_unstructured_ingest/test-ingest-azure.sh
and witness output .json's are visible. Yay! Now, to instead clean up
output .json's and workdir, run:
UNSTRUCTURED_CLEANUP_DEV_FIXTURES=1 bash -x
./test_unstructured_ingest/test-ingest-azure.sh
and witness the files have been cleaned up. Yay!
