unstructured

mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-12-12 07:34:09 +00:00

History

feat: modify test-ingest-src and evaluation-metrics to allow EXPORT_DIR (#2551 )

The current `test-ingest-src.sh` and `evaluation-metrics` do not allow
passing the `EXPORT_DIR` (`OUTPUT_ROOT` in `evaluation-metrics`). It is
currently saving at the current working directory
(`unstructured/test_unstructured_ingest`). When running the eval from
`core-product`, all outputs is now saved at
`core-product/upstream-unstructured/test_unstructured_ingest` which is
undesirable.

This PR modifies two scripts to accommodate such behavior:
1. `test-ingest-src.sh` - assign `EVAL_OUTPUT_ROOT` to the value set
within the environment if exist, or the current working directory if
not. Then calls to run `evaluation-metrics.sh`.
2. `evaluation-metrics.sh` - accepting param from `test-ingest-src.sh`
if exist, or to the value set within the environment if exist, or the
current directory if not.

(Note: I also add param to `evaluation-metrics.sh` because it makes
sense to allow a separate run to be able to specify an export directory)

This PR should work in sync with another PR under `core-product`, which
I will add the link here later.

**To test:**

Run the script below, change `$SCRIPT_DIR` as needed to see the result.

```
export OVERWRITE_FIXTURES=true

./upstream-unstructured/test_unstructured_ingest/src/s3.sh

SCRIPT_DIR=$(dirname "$(realpath "$0")")
bash -x ./upstream-unstructured/test_unstructured_ingest/evaluation-metrics.sh text-extraction "$SCRIPT_DIR"
```

----

This PR also updates the requirements by `make pip-compile` since the
`click` module was not found.

2024-02-17 05:21:15 +00:00

dest

fix: change opensearch port (#2517 )

2024-02-07 21:25:04 +00:00

example-docs

roman/cli infer table arg (#1685 )

2023-10-12 16:14:53 +00:00

expected-structured-output

fix(xlsx): xlsx subtable algorithm (#2534 )

2024-02-13 20:29:17 -08:00

failed-partition-docs

roman/ingest continue on error (#1736 )