yujunjun/olmocr
1
0
Fork 0
mirror of https://github.com/allenai/olmocr.git synced 2025-10-11 08:12:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

Cleaning up some training requirements installation steps

Browse Source
This commit is contained in:
Jake Poznanski 2025-08-05 19:42:46 +00:00
parent c9b8088bc6
commit 8b8c6bb837
4 changed files with 3 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
gantry-train-requirements.txt
View File

@ -1,6 +0,0 @@
torchvision
accelerate
peft
wandb
omegaconf
einops

2
olmocr/train/README.md
View File

@ -9,7 +9,7 @@ The first step is to setup your python/conda environment, and set things up the
Then, add in some extra training requirements: Then, add in some extra training requirements:
```bash ```bash
pip install -r gantry-train-requirements.txt pip install .[train]
pip install transformers==4.52.4 pip install transformers==4.52.4
pip install flash-attn==2.8.0.post2 --no-build-isolation pip install flash-attn==2.8.0.post2 --no-build-isolation
``` ```

5
pyproject.toml
View File

@ -99,13 +99,10 @@ train = [
"torch", "torch",
"torchvision", "torchvision",
"accelerate", "accelerate",
"datasets",
"peft", "peft",
"wandb", "wandb",
"omegaconf", "omegaconf",
"s3fs", "einops"
"necessary",
"einops",
"augraphy", "augraphy",
] ]

2
scripts/train/newtrainer-beaker.sh
View File

@ -92,7 +92,7 @@ b = Beaker.from_env(default_workspace="ai2/olmocr")
# Build the training command # Build the training command
commands = [ commands = [
"pip install -r gantry-train-requirements.txt", "pip install .[train]",
"pip install transformers==4.52.4", "pip install transformers==4.52.4",
"pip install flash-attn==2.8.0.post2 --no-build-isolation", "pip install flash-attn==2.8.0.post2 --no-build-isolation",
"pip install s5cmd", "pip install s5cmd",