yujunjun/olmocr
1
0
Fork 0
mirror of https://github.com/allenai/olmocr.git synced 2025-12-27 07:05:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

update args; include output

Browse Source
This commit is contained in:
kyleclo 2025-02-13 17:06:36 -08:00
parent 88c18b3afa
commit a790ba73ee
1 changed files with 20 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
scripts/elo/calculate_elo_ratings.py
View File

@ -13,7 +13,26 @@ See data at scripts/elo/ratings.csv
gotocr_format,mineru,38,37,50.7,49.3
Invoke via
python calculate_elo_ratings.py ratings.csv --num-bootstrap 1000 --num-elo-sims 10 --confidence-level 95 --seed 123
python calculate_elo_ratings.py ratings.csv --num-bootstrap 5000 --num-elo-sims 100 --confidence-level 95 --seed 123
Output:
Bootstrapped Elo Ratings (95% CI):
--------------------------------------------------
pdelf 1813.0 ± 84.9 [1605.9, 1930.0]
mineru 1545.2 ± 99.7 [1336.7, 1714.1]
marker 1429.1 ± 100.7 [1267.6, 1645.5]
gotocr_format 1212.7 ± 82.0 [1097.3, 1408.3]
Pairwise Significance Tests:
--------------------------------------------------
gotocr_format vs marker Δ = -216.3 [-470.8, 135.0] p = 0.218
gotocr_format vs mineru Δ = -332.5 [-567.5, 19.3] p = 0.051
gotocr_format vs pdelf Δ = -600.3 [-826.1, -344.3] p = 0.000*
marker vs mineru Δ = -116.1 [-365.4, 246.5] p = 0.430
marker vs pdelf Δ = -383.9 [-610.6, -10.9] p = 0.044*
mineru vs pdelf Δ = -267.8 [-517.3, 104.0] p = 0.135
@kylel
@ -25,7 +44,6 @@ from itertools import combinations
import click
import numpy as np
import pandas as pd
from scipy import stats
from tqdm import tqdm