unstructured/test_unstructured_ingest/metrics/all-docs-cct.tsv

filename	connector	cct-accuracy	cct-%missing
handbook-1p.docx	box	0.974	0.03
example-10k.html	local	0.686	0.04
IRS-form-1987.pdf	azure	0.783	0.13
build: text extraction evaluation metrics workflow added (#1757) Executive Summary This PR adds the evaluation metrics to our current workflow. It verifies the flow that when the code is pushed, the code will gets evaluate against our gold standard and output into `.tsv` file. Technical Details - Adds evaluation metrics to the test-ingest workflow - Make use of `structured-output` from `test-ingest` and compare to the gold-standard uploaded in s3, and download into local when make comparison. The current folder in-use is `s3://utic-dev-tech-fixtures/small-cct`. This dir is editable in the shell script. - With this PR, only one file from one connector is use to compare. Misc - Not many overlapped files between test-ingest and gold-standard. More files will be added. Outputs 2 `.tsv` files are saved under `test_unstructured_ingest/metrics/`. ![image](https://github.com/Unstructured-IO/unstructured/assets/2177850/222e437c-1a94-4d7c-9320-81696633b1ae) ![image](https://github.com/Unstructured-IO/unstructured/assets/2177850/5c840322-6739-4634-8868-eba04b4ebc96) --------- Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: Klaijan <Klaijan@users.noreply.github.com> 2023-10-23 17:39:22 -04:00			`filename connector cct-accuracy cct-%missing`
refactor: ci workflow refactor (#1907) Refactor the evaluation scripts including `unstructured/ingest/evaluation.py` `test_unstructured_ingest/evaluation-metrics.sh` for more structured code and usage. - The script is now only use one python script call with param - Adds function to build string for output_args (`--output_dir --output_list) and source_args (`--source_dir --source_args`) - Now accepts evaluation to call as a param, currently only accepts `text-extraction` and `element-type` Example to call the function: ```sh evaluation-metrics.sh text-extraction``` --------- Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: Klaijan <Klaijan@users.noreply.github.com> 2023-11-01 11:58:23 -04:00			`handbook-1p.docx box 0.974 0.03`
feat: clean pdfminer elements inside tables (#1808) This PR introduces `clean_pdfminer_inner_elements` , which deletes pdfminer elements inside other detection origins such as YoloX or detectron. This function returns the clean document. Also, the ingest-test fixtures were updated to reflect the new standard output. The best way to check that this function is working properly is check the new test `test_clean_pdfminer_inner_elements` in `test_unstructured/partition/utils/test_processing_elements.py` --------- Co-authored-by: Roman Isecke <roman@unstructured.io> Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: rbiseck3 <rbiseck3@users.noreply.github.com> Co-authored-by: Roman Isecke <136338424+rbiseck3@users.noreply.github.com> 2023-10-30 01:10:51 -06:00			`example-10k.html local 0.686 0.04`
refactor: ci workflow refactor (#1907) Refactor the evaluation scripts including `unstructured/ingest/evaluation.py` `test_unstructured_ingest/evaluation-metrics.sh` for more structured code and usage. - The script is now only use one python script call with param - Adds function to build string for output_args (`--output_dir --output_list) and source_args (`--source_dir --source_args`) - Now accepts evaluation to call as a param, currently only accepts `text-extraction` and `element-type` Example to call the function: ```sh evaluation-metrics.sh text-extraction``` --------- Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: Klaijan <Klaijan@users.noreply.github.com> 2023-11-01 11:58:23 -04:00			`IRS-form-1987.pdf azure 0.783 0.13`