yujunjun/olmocr
1
0
Fork 0
mirror of https://github.com/allenai/olmocr.git synced 2025-10-11 16:22:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,444 Commits 73 Branches 30 Tags
Commit Graph

229 Commits

Author SHA1 Message Date
Jake Poznanski
2fca448105 Using new budget code 2025-08-06 16:31:08 +00:00
Jake Poznanski
8b8c6bb837 Cleaning up some training requirements installation steps 2025-08-05 19:42:46 +00:00
Jake Poznanski
c9b8088bc6 Adding some preempt flags 2025-08-05 18:00:46 +00:00
Jake Poznanski
55f8ba0ac0 Fixing configs 2025-08-04 22:54:39 +00:00
Jake Poznanski
12f8a90f1b Copying preprocessed files to local ssd in trainer script 2025-08-04 22:18:38 +00:00
Jake Poznanski
7c098955a9 Trying fix for transformers benchmark 2025-08-04 19:50:05 +00:00
Jake Poznanski
df52cb0e0e Small fixes for transformers test runner 2025-07-25 03:18:24 +00:00
Jake Poznanski
cf1912dec4 Some transformer bench ideas 2025-07-24 21:20:15 +00:00
Jake Poznanski
16145a4b32 Need accelerate 2025-07-16 18:51:37 +00:00
Jake Poznanski
31c834dcdd Constants 2025-07-16 02:15:17 +00:00
Jake Poznanski
5ea4e8a6e2 Compare vllm script 2025-07-15 22:55:49 +00:00
Jake Poznanski
24608956a0 Working on comparing to vllm 2025-07-15 22:21:54 +00:00
Jake Poznanski
e6c98236b6 Adding more pipeline retry stats, compress code fixed 2025-07-15 21:41:10 +00:00
Jake Poznanski
4dbbf91e1c Compression script 2025-07-15 21:26:15 +00:00
Jake Poznanski
1092213c5f Merge branch 'jakep/new_traininer_nojson_newprompt' into jakep/new_trainer 2025-07-15 17:44:55 +00:00
Jake Poznanski
43ae28dde4 Prepare checkpoint works for older models too 2025-07-14 21:30:32 +00:00
Jake Poznanski
f014c2aaf9 Need to reserve all 8 gpus for reliable performance benchmark, even if you only use 1 2025-07-14 21:02:14 +00:00
Jake Poznanski
65d0edcaae Adding guided decoding option 2025-07-10 15:13:26 +00:00
Jake Poznanski
a7e2f719bf Start a preemptible one at least once 2025-07-02 19:26:30 +00:00
Jake Poznanski
79a7818517 New trainer launch script for beaker 2025-07-01 01:43:38 +00:00
Jake Poznanski
dcf026a63c Better script 2025-07-01 01:40:55 +00:00
Jake Poznanski
9f0f912101 Ugh 2025-06-30 23:37:35 +00:00
Jake Poznanski
1d007d1bf2 Perhaps fixing default config 2025-06-30 22:58:21 +00:00
Jake Poznanski
e7020c7f50 More configs 2025-06-30 22:49:42 +00:00
Jake Poznanski
5c2d69a3d7 Some cleanup stuff 2025-06-30 21:24:35 +00:00
Jake Poznanski
656dbef833 Frontier configs 2025-06-30 17:43:30 +00:00
Jake Poznanski
e2f2d36e4f More typos 2025-06-30 17:41:19 +00:00
Jake Poznanski
55a737ca6b script 2025-06-30 17:32:01 +00:00
Jake Poznanski
ba49fd53d9 frontier train script let's see what happens 2025-06-30 17:30:17 +00:00
Jake Poznanski
b96454b786 Merge branch 'main' into jakep/new_trainer 2025-06-27 16:21:58 +00:00
Jake Poznanski
1451dd1395 weka 2025-06-27 02:57:26 +00:00
Jake Poznanski
dee3730231 Gantry stuff 2025-06-26 18:34:53 +00:00
Jake Poznanski
0d7836b111 Basic atttempt to run trainer script 2025-06-25 23:22:59 +00:00
Jake Poznanski
d7e5037192 New trainer launch script cleanups 2025-06-25 23:05:32 +00:00
Jake Poznanski
ec5c5b6444 Updating pareto plots 2025-06-17 21:41:23 +00:00
Jake Poznanski
6c51829ae6 Some helper scripts 2025-06-17 21:21:50 +00:00
Jake Poznanski
1295e171bb Merge branch 'main' of https://github.com/allenai/olmocr 2025-06-12 22:35:09 +00:00
Jake Poznanski
37090e2801 Go back to workers 1 in marker test script 2025-06-12 22:35:08 +00:00
Jake Poznanski
af02f15f24
Merge pull request #236 from VikParuchuri/main 
Fix marker benchmarks
 2025-06-12 15:24:17 -07:00
Jake Poznanski
3da6e2d587 Pareto plot update, keep cost the same for now 2025-06-12 22:23:41 +00:00
Jake Poznanski
fcd8bbec92 Install aws cli 2025-06-12 21:38:28 +00:00
Jake Poznanski
fc06797bec aws cli 2025-06-12 21:29:39 +00:00
Jake Poznanski
59e0a1ccb0 Marker wants newer torchvision 2025-06-12 21:23:53 +00:00
Jake Poznanski
0f3b45c1a3 Add time 2025-06-12 21:19:17 +00:00
Jake Poznanski
4bfcfce767 Actually install the right thing 2025-06-12 21:18:58 +00:00
Jake Poznanski
f8dfd85765 Script 2025-06-12 21:13:31 +00:00
Jake Poznanski
044874a634 Adding marker benchmark 2025-06-12 21:12:58 +00:00
Jake Poznanski
9787d007b9 Pulling in bigger benchmark script from vllm branch to main 2025-06-12 21:02:46 +00:00
Jake Poznanski
43c94fea58 Bencharmk update 2025-06-12 20:47:58 +00:00
Jake Poznanski
b1e064f8a6 Run benchmark script will also start a job to convert 10k docs from olmocr-mix to check performance 2025-06-12 20:27:50 +00:00