511 Commits

Author SHA1 Message Date
Jake Poznanski
d0eea81c00 Dealing with issue with molmo unused params 2025-01-24 16:27:42 +00:00
Jake Poznanski
dabecd9ef0 More configs 2025-01-23 23:39:56 +00:00
Jake Poznanski
858b49656f Getting ready to train molmo 4096 context 2025-01-23 15:32:04 -08:00
Jake Poznanski
f42bb02fce Manually adding gradient checkpointing 2025-01-23 15:18:22 -08:00
Jake Poznanski
18569a4c63 Adding molmo code locally 2025-01-23 15:18:00 -08:00
Jake Poznanski
01469af463 Doing some debugging 2025-01-23 10:58:43 -08:00
Jake Poznanski
201fec3ad9 Config update 2025-01-23 02:48:30 +00:00
Jake Poznanski
72d2fa2fd4 Reviewing molmo training 2025-01-22 15:23:08 -08:00
Jake Poznanski
0311b445fd Some small updates 2025-01-21 23:01:30 +00:00
Jake Poznanski
6586744718 Building some data summary tools 2025-01-21 22:38:31 +00:00
Jake Poznanski
c74e3d1440 ELO stuff 2025-01-16 18:00:12 +00:00
Jake Poznanski
18f72b4e1b New ELO building stuff finished up I think 2025-01-16 00:22:29 +00:00
Jake Poznanski
50464c1057 build elo v1 2025-01-15 23:35:18 +00:00
Jake Poznanski
3a28955857 Added ELO scores 2025-01-14 22:57:57 +00:00
Jake Poznanski
a8d9a55fdb Fixes for elo 2025-01-14 22:57:17 +00:00
Jake Poznanski
00f2a67ac4 More elo scoring stuff 2025-01-14 22:40:56 +00:00
Jake Poznanski
834e91c8d5 runelo start 2025-01-14 21:08:23 +00:00
Jake Poznanski
ef4167dc45 Test set script 2025-01-14 19:36:18 +00:00
Jake Poznanski
683be68707 Better error handling on expand_s3_glob 2025-01-10 20:33:24 +00:00
Jake Poznanski
5e633e025a Merge branch 'main' of https://github.com/allenai/pdelfin 2025-01-10 19:38:44 +00:00
Jake Poznanski
0d1fc08081 Small fixes 2025-01-10 19:38:42 +00:00
Jake Poznanski
02ec972e41
Update README.md 2025-01-10 09:45:40 -08:00
Jake Poznanski
2190f6117e Merge branch 'main' of https://github.com/allenai/pdelfin 2024-12-10 17:18:12 +00:00
Jake Poznanski
e2bbd0eec9 Adding some long context stats 2024-12-10 17:18:10 +00:00
Jake Poznanski
5692a76350 Ok, direct easy test for diffs now 2024-12-04 13:27:51 -08:00
Jake Poznanski
a56ce71771 Merge branch 'main' of https://github.com/allenai/pdelfin into main 2024-12-04 13:20:14 -08:00
Jake Poznanski
48f3ab82bd Working on some random tests 2024-12-04 13:20:10 -08:00
Jake Poznanski
0b72eda794 Move form check into exception handler, don't mark the work item as done if it had an exception on it 2024-12-04 19:08:21 +00:00
Jake Poznanski
fa318dac7c New version with s3 fix in it 2024-12-04 18:46:39 +00:00
Jake Poznanski
84c53c27d2 Merge branch 'main' of https://github.com/allenai/pdelfin 2024-12-04 17:56:47 +00:00
Jake Poznanski
e9c3c21731 Skipping files which are not found 2024-12-04 17:56:45 +00:00
Jake Poznanski
7bec33a67a Merge branch 'main' of https://github.com/allenai/pdelfin into main 2024-12-03 15:32:55 -08:00
Jake Poznanski
917cdeccba Some more tests 2024-12-03 15:32:53 -08:00
Jake Poznanski
3e33ce1cde Ignores 2024-12-03 18:50:31 +00:00
Jake Poznanski
37cdb9e146 Merge branch 'main' of https://github.com/allenai/pdelfin 2024-12-03 18:49:41 +00:00
Jake Poznanski
1eda300e01 Dolma viewer niceties 2024-12-03 18:49:16 +00:00
Jake Poznanski
535181d8e0 Moving to manual HTTP Post, have had succss with 10k page files now 2024-12-03 10:48:52 -08:00
Jake Poznanski
fe04db828e Better error handling 2024-12-02 23:56:45 +00:00
Jake Poznanski
35502bc6f2 Limit the number of retries on the server process 2024-12-02 23:46:46 +00:00
Jake Poznanski
b3ca86a182 More robust to errors when reading logs which had caused freezes 2024-12-02 13:59:42 -08:00
Jake Poznanski
d4f3cfff4d More reliable weka 2024-11-27 19:11:20 +00:00
Jake Poznanski
687210572d Merge branch 'main' of https://github.com/allenai/pdelfin 2024-11-26 22:22:38 +00:00
Jake Poznanski
c93fc36f72 Missing import 2024-11-26 22:22:36 +00:00
Jake Poznanski
9b9d04c8e9 aaa 2024-11-26 08:38:25 -08:00
Jake Poznanski
386374bd72 More prints 2024-11-25 16:08:24 -08:00
Jake Poznanski
04d6123037 Doing some experiments 2024-11-25 15:36:04 -08:00
Jake Poznanski
51614efc83 More log probs investigation 2024-11-25 11:24:21 -08:00
Jake Poznanski
28d52602e9 More test code 2024-11-25 11:00:03 -08:00
Jake Poznanski
606e81bfea Not happy here with this test 2024-11-25 10:32:18 -08:00
Jake Poznanski
d7838372e8 Full test 2024-11-25 10:25:55 -08:00