Nick Franck a040483a7e

Add OCR_AGENT_CACHE_SIZE environment variable (#4066) ...

## Problem
OCR agents used unlimited caching, causing excessive memory usage. Each
cached OCR agent consumes different amounts of memory, but can easily
consume ~800MB.

## Solution
Add `OCR_AGENT_CACHE_SIZE` environment variable to limit cached OCR
agents per process.

- **Default**: 1 cached agent
- **Configurable**: Set to 0 to disable caching, or higher for more
languages

2025-07-18 15:47:55 +00:00

..

chunking

fix chucking text None type has no attribute stripe (#4018)

2025-06-12 18:28:46 +00:00

cleaners

Luke/CVE bump (#3928)

2025-02-19 17:23:31 +00:00

common

chunk: relax table segregation during chunking (#3812)

2024-12-09 18:57:22 +00:00

documents

feat: keep input tag's class attr in table (#4064)

2025-07-16 21:46:58 +00:00

embed

Refactoring VoyageAI integration (#3878)

2025-01-28 21:45:40 +00:00

file_utils

feat: added new exception type to epub conversions (#4052)

2025-07-15 10:56:22 +00:00

metrics

fix: add empty string case for language metadata (#4062)

2025-07-16 21:35:00 +00:00

models

chore: Remove PDF parsing code and dependencies (#75)

2022-11-21 11:47:29 -06:00

nlp

bugfix/fix ndjson detection (#3905)

2025-02-11 14:21:28 +00:00

partition

Add OCR_AGENT_CACHE_SIZE environment variable (#4066)

2025-07-18 15:47:55 +00:00

patches

fix: Correctly patch pdfminer to avoid unnecessarily and unsuccessfully repairing PDFs with long content streams, causing needless and endless OCR (#3822)

2025-01-24 14:27:25 -06:00

staging

feat: convert elements to markdown (#4055)

2025-07-16 14:34:29 +00:00

__init__.py

feat: add GLOBAL_WORKING_DIR and GLOBAL_WORKING_PROCESS_DIR config parameteres (#3014)

2024-05-17 19:16:10 +00:00

__version__.py

Add OCR_AGENT_CACHE_SIZE environment variable (#4066)

2025-07-18 15:47:55 +00:00

errors.py

feat: detect language for PDFs (#4051)

2025-07-15 18:53:28 +00:00

logger.py

Add basic call to scarf to get anonymous analytics (#1705)

2023-10-11 09:15:36 -07:00

py.typed

chore: add py.typed (#3043)

2024-05-20 08:52:04 -04:00

utils.py

chore: utils update (#3909)

2025-02-07 05:58:23 +00:00