unstructured/test_unstructured_ingest/metrics/text-extraction/aggregate-scores-cct.tsv

metric	average	sample_sd	population_sd	count
cct-accuracy	0.809	0.24	0.233	17
cct-%missing	0.026	0.033	0.032	17
refactor: measure_text_edit_distance function for aggregation (#2108) - Refactor `metrics/evaluation.py` to accepts `grouping` as parameter. - Switch to `DataFrame` for easier analysis and aggregation. 2023-11-22 16:30:16 -05:00			`metric average sample_sd population_sd count`
feat(ingest): add basic chunking to ingest (#2380) The new "basic" chunking strategy and overlap options need to be available from the ingest CLI. An ingest test of those features is also welcome, both to verify the ingest feature and to defend against regressions in the chunking code. Add a local ingest test exercising both the "basic" chunking strategy and intra-chunk overlap. Since there is no new source connector involved, use the local ingest source and destination. Update documentation to suit, filling in some details that hadn't made it into the docs yet. 2024-01-12 12:27:34 -08:00			`cct-accuracy 0.809 0.24 0.233 17`
fix: default hi_res model rely on inference setting (#2441) - there are multiple places setting the default `hi_res_model_name` in both `unstructured` and `unstructured-inference` - they lead to inconsistency and unexpected behaviors - this fix removes a helper in `unstructured` that tries to set the default hi_res layout detection model; instead we rely on the `unstructured-inference` to provide that default when no explicit model name is passed in ## test ```bash UNSTRUCTURED_INCLUDE_DEBUG_METADATA=true ipython ``` ```python from unstructured.partition.auto import partition # find a pdf file elements = partition("foo.pdf", strategy="hi_res") assert elements[0].metadata.detection_origin == "yolox" ``` --------- Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: badGarnet <badGarnet@users.noreply.github.com> 2024-01-29 10:44:41 -06:00			`cct-%missing 0.026 0.033 0.032 17`