yujunjun/olmocr
1
0
Fork 0
mirror of https://github.com/allenai/olmocr.git synced 2025-11-03 11:35:29 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #341 from charitarthchugh/charitarthchugh/vllm-defaults-speedup

Browse Source 
Add chunked prefill and limit mm per prompt options
This commit is contained in:
Jake Poznanski 2025-10-06 13:23:47 -07:00 committed by GitHub
commit 2b70b50312
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
olmocr/pipeline.py
View File

@ -636,6 +636,8 @@ async def vllm_server_task(model_name_or_path, args, semaphore, unknown_args=Non
str(args.tensor_parallel_size), str(args.tensor_parallel_size),
"--data-parallel-size", "--data-parallel-size",
str(args.data_parallel_size), str(args.data_parallel_size),
"--enable-chunked-prefill",
"--limit-mm-per-prompt '{\"video\": 0}'"
] ]
if args.gpu_memory_utilization is not None: if args.gpu_memory_utilization is not None: