582 Commits

Author SHA1 Message Date
Jake Poznanski
dee494ad7b Local file stuff 2025-01-28 15:12:28 -08:00
Jake Poznanski
7882944e88 Local pdf support 2025-01-28 15:03:31 -08:00
Jake Poznanski
dbe54871db Support stats feature later 2025-01-28 14:29:46 -08:00
Jake Poznanski
48447b616c Can use remote s3 files, and local workspace now 2025-01-28 14:28:19 -08:00
Jake Poznanski
50f9a6adb5 Name refactor 2025-01-28 14:16:53 -08:00
Jake Poznanski
e0afb935fa Better check for separate sglang installation step 2025-01-28 13:56:00 -08:00
Jake Poznanski
00e3aac058 Inference test for qwen2 and 2.5, work queue fixes, build current still broken 2025-01-27 15:58:48 -08:00
Jake Poznanski
4d0d9246b4 Merge branch 'main' of https://github.com/allenai/olmocr 2025-01-27 21:11:43 +00:00
Jake Poznanski
b28aad61bb More test docs 2025-01-27 21:11:23 +00:00
Jake Poznanski
43c3caa559 Merge branch 'main' of https://github.com/allenai/olmocr into main 2025-01-27 12:59:51 -08:00
Jake Poznanski
4baba92c29 Remove some todos 2025-01-27 12:59:43 -08:00
Jake Poznanski
96ae2dd49b Refactoring 2025-01-27 20:45:28 +00:00
Jake Poznanski
c6062677aa Cleaning up some unused code 2025-01-27 18:48:15 +00:00
Jake Poznanski
d8c13d05f6 Readmes and version updates 2025-01-27 18:41:13 +00:00
Jake Poznanski
b2894d0280 Massive refactor from pdelfin to olmocr 2025-01-27 18:30:41 +00:00
Jake Poznanski
7261bfc0b9
Update README.md 2025-01-27 10:21:59 -08:00
Jake Poznanski
cbfc80355a
Merge pull request #27 from allenai/molmo
Molmo
2025-01-27 10:19:10 -08:00
Jake Poznanski
ad88a82ee9 more elos 2025-01-27 17:16:21 +00:00
Jake Poznanski
5b429ad100 Higher lr for molmo, fixed evals 2025-01-24 23:15:35 +00:00
Jake Poznanski
d0eea81c00 Dealing with issue with molmo unused params 2025-01-24 16:27:42 +00:00
Jake Poznanski
dabecd9ef0 More configs 2025-01-23 23:39:56 +00:00
Jake Poznanski
aa59d38a5b Merge branch 'main' of https://github.com/allenai/pdelfin 2025-01-23 23:32:56 +00:00
Jake Poznanski
eacd0442c4 csv output 2025-01-23 23:32:54 +00:00
Jake Poznanski
858b49656f Getting ready to train molmo 4096 context 2025-01-23 15:32:04 -08:00
Jake Poznanski
f42bb02fce Manually adding gradient checkpointing 2025-01-23 15:18:22 -08:00
Jake Poznanski
18569a4c63 Adding molmo code locally 2025-01-23 15:18:00 -08:00
Jake Poznanski
01469af463 Doing some debugging 2025-01-23 10:58:43 -08:00
Jake Poznanski
201fec3ad9 Config update 2025-01-23 02:48:30 +00:00
Jake Poznanski
72d2fa2fd4 Reviewing molmo training 2025-01-22 15:23:08 -08:00
Jake Poznanski
0311b445fd Some small updates 2025-01-21 23:01:30 +00:00
Jake Poznanski
6586744718 Building some data summary tools 2025-01-21 22:38:31 +00:00
Jake Poznanski
c74e3d1440 ELO stuff 2025-01-16 18:00:12 +00:00
Jake Poznanski
18f72b4e1b New ELO building stuff finished up I think 2025-01-16 00:22:29 +00:00
Jake Poznanski
50464c1057 build elo v1 2025-01-15 23:35:18 +00:00
Jake Poznanski
3a28955857 Added ELO scores 2025-01-14 22:57:57 +00:00
Jake Poznanski
a8d9a55fdb Fixes for elo 2025-01-14 22:57:17 +00:00
Jake Poznanski
00f2a67ac4 More elo scoring stuff 2025-01-14 22:40:56 +00:00
Jake Poznanski
834e91c8d5 runelo start 2025-01-14 21:08:23 +00:00
Jake Poznanski
ef4167dc45 Test set script 2025-01-14 19:36:18 +00:00
Jake Poznanski
683be68707 Better error handling on expand_s3_glob 2025-01-10 20:33:24 +00:00
Jake Poznanski
5e633e025a Merge branch 'main' of https://github.com/allenai/pdelfin 2025-01-10 19:38:44 +00:00
Jake Poznanski
0d1fc08081 Small fixes 2025-01-10 19:38:42 +00:00
Jake Poznanski
02ec972e41
Update README.md 2025-01-10 09:45:40 -08:00
Jake Poznanski
2190f6117e Merge branch 'main' of https://github.com/allenai/pdelfin 2024-12-10 17:18:12 +00:00
Jake Poznanski
e2bbd0eec9 Adding some long context stats 2024-12-10 17:18:10 +00:00
Jake Poznanski
5692a76350 Ok, direct easy test for diffs now 2024-12-04 13:27:51 -08:00
Jake Poznanski
a56ce71771 Merge branch 'main' of https://github.com/allenai/pdelfin into main 2024-12-04 13:20:14 -08:00
Jake Poznanski
48f3ab82bd Working on some random tests 2024-12-04 13:20:10 -08:00
Jake Poznanski
0b72eda794 Move form check into exception handler, don't mark the work item as done if it had an exception on it 2024-12-04 19:08:21 +00:00
Jake Poznanski
fa318dac7c New version with s3 fix in it 2024-12-04 18:46:39 +00:00