yujunjun/olmocr
1
0
Fork 0
mirror of https://github.com/allenai/olmocr.git synced 2025-06-27 04:00:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
30 Commits 38 Branches 16 Tags
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Jake Poznanski dc71b28ddd No need to save tokenizer
2024-09-23 10:06:04 -07:00
.github
Running personalize script on template
2024-09-17 15:06:59 +00:00
docs
Running personalize script on template
2024-09-17 15:06:59 +00:00
pdelfin
No need to save tokenizer
2024-09-23 10:06:04 -07:00
scripts
Running personalize script on template
2024-09-17 15:06:59 +00:00
tests
Okay, reasonably happy with the dataprep pipeline
2024-09-20 13:04:47 -07:00
.dockerignore
Initial commit
2024-09-17 07:53:43 -07:00
.gitignore
Hoping to get a basic hf Trainer to run
2024-09-20 15:53:11 -07:00
.readthedocs.yaml
Initial commit
2024-09-17 07:53:43 -07:00
CHANGELOG.md
Initial commit
2024-09-17 07:53:43 -07:00
LICENSE
Initial commit
2024-09-17 07:53:43 -07:00
Makefile
Running personalize script on template
2024-09-17 15:06:59 +00:00
pyproject.toml
Smaller config for now, fixing a few requirements
2024-09-23 08:20:08 -07:00
README.md
Running personalize script on template
2024-09-17 15:06:59 +00:00
RELEASE_PROCESS.md
Running personalize script on template
2024-09-17 15:06:59 +00:00

README.md

pdelfin

Description
Toolkit for linearizing PDFs for LLM datasets/training
Readme Apache-2.0 349 MiB
Languages
Python 91.7%
HTML 5.6%
Shell 2.5%
Dockerfile 0.2%