Jake Poznanski 4eddb1b45f Okay, reasonably happy with the dataprep pipeline

2024-09-20 13:04:47 -07:00

..

gnarly_pdfs

450tok/sec/core with smollm that appears to work well

2024-09-17 19:59:02 +00:00

__init__.py

Initial commit

2024-09-17 07:53:43 -07:00

test_coherency.py

Formatting

2024-09-18 22:52:42 +00:00

test_dataloader.py

Formatting

2024-09-18 22:52:42 +00:00

test_dataprep.py

Okay, reasonably happy with the dataprep pipeline

2024-09-20 13:04:47 -07:00

test_filter.py

Formatting

2024-09-18 22:52:42 +00:00