yujunjun/olmocr
1
0
Fork 0
mirror of https://github.com/allenai/olmocr.git synced 2025-09-25 08:20:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
99 Commits 68 Branches 25 Tags
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Jake Poznanski b4e9d6a2b8 Buildsilver script suppors reservoir sampling so it can sample 100M+ paths now efficiently
2024-09-30 18:41:18 +00:00
.github
Running personalize script on template
2024-09-17 15:06:59 +00:00
docs
Running personalize script on template
2024-09-17 15:06:59 +00:00
pdelfin
Buildsilver script suppors reservoir sampling so it can sample 100M+ paths now efficiently
2024-09-30 18:41:18 +00:00
scripts
Fixing the refiner input prompt to something simpler that doesn't depend on the training data. Fixing beaker job workspace and bumping priority to high.
2024-09-27 22:54:07 +00:00
tests
Fixing the refiner input prompt to something simpler that doesn't depend on the training data. Fixing beaker job workspace and bumping priority to high.
2024-09-27 22:54:07 +00:00
.dockerignore
Initial commit
2024-09-17 07:53:43 -07:00
.gitignore
Hoping to get a basic hf Trainer to run
2024-09-20 15:53:11 -07:00
.readthedocs.yaml
Initial commit
2024-09-17 07:53:43 -07:00
CHANGELOG.md
Initial commit
2024-09-17 07:53:43 -07:00
gantry-requirements.txt
bnb
2024-09-26 03:30:35 +00:00
LICENSE
Initial commit
2024-09-17 07:53:43 -07:00
Makefile
Running personalize script on template
2024-09-17 15:06:59 +00:00
pyproject.toml
Hopefuly will train now
2024-09-27 15:16:12 +00:00
README.md
Running personalize script on template
2024-09-17 15:06:59 +00:00
RELEASE_PROCESS.md
Running personalize script on template
2024-09-17 15:06:59 +00:00
sequence_lengths_histogram.png
Sampling some sequence lengths
2024-09-25 09:05:11 -07:00

README.md

pdelfin

Description
Toolkit for linearizing PDFs for LLM datasets/training
Readme Apache-2.0 361 MiB
Languages
Python 90.1%
HTML 5.7%
Shell 4%
Dockerfile 0.2%