unstructured/test_unstructured/metrics/test_table_alignment.py
Pluto 5d89b41b1a
Fix not counting false negatives and false positives in table metrics (#3300)
This pull request fixes counting tables metric for three cases:
- False Negatives: when table exist in ground truth but any of the
predicted tables doesn't match the table, the table should count as 0
and the file should not be completely skipped (before it was np.NaN).
- False Positives: When there is a predicted table that didn't match any
ground truth table it should be counted as 0, right now it is skipped in
processing (matched_indices==-1)
- The file should be completely skipped only if there is no tables in
ground truth and in prediction

In short we can say that previous metric calculation didn't consider OD
mistakes
2024-07-02 10:07:24 +00:00

15 lines
554 B
Python

from unstructured.metrics.table.table_alignment import TableAlignment
def test_get_element_level_alignment_when_no_match():
example_table = [{"row_index": 0, "col_index": 0, "content": "a"}]
metrics = TableAlignment.get_element_level_alignment(
predicted_table_data=[example_table],
ground_truth_table_data=[example_table],
matched_indices=[-1],
)
assert metrics["col_index_acc"] == 0
assert metrics["row_index_acc"] == 0
assert metrics["row_content_acc"] == 0
assert metrics["col_content_acc"] == 0