yujunjun/unstructured
1
0
Fork 0
mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-11-09 23:17:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: correctly install mesa-gl for arm (#3647)

Browse Source 
### Summary

Fixes the `arm64` image builds, which will be available again starting
in version `0.15.13`. A fix was implemented upstream in
https://github.com/Unstructured-IO/base-images/pull/47 and a workaround
that installed `x86` packages in the `unstructured` repo was removed.

### Testing

See [this
job](https://github.com/Unstructured-IO/unstructured/actions/runs/10948943594/job/30401108059?pr=3647)
for a successful `arm64` build on the feature branch.
This commit is contained in:
Matt Robinson 2024-09-20 09:32:47 -04:00 committed by GitHub
parent 0ed69a1ac3
commit 7d66a236f1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
36 changed files with 198 additions and 156 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGELOG.md
View File

@ -1,4 +1,4 @@
## 0.15.13-dev3 ## 0.15.13
### Enhancements ### Enhancements
@ -9,6 +9,7 @@
### Fixes ### Fixes
* **Fixes high memory overhead for intersection area computation** Using `numpy.float32` for coordinates and remove intermediate variables to reduce memory usage when computing intersection areas * **Fixes high memory overhead for intersection area computation** Using `numpy.float32` for coordinates and remove intermediate variables to reduce memory usage when computing intersection areas
* **Fixes the `arm64` image build** `arm64` builds are now fixed and will be available against starting with the `0.15.13` release.
## 0.15.12 ## 0.15.12

11
Dockerfile
View File

@ -9,17 +9,6 @@ COPY unstructured unstructured
COPY test_unstructured test_unstructured COPY test_unstructured test_unstructured
COPY example-docs example-docs COPY example-docs example-docs
# NOTE(robinson) - temporary workaround to install mesa-gl 24.1.0 because
# libgallum is missing in mesa-gl 24.2.0 from the wolfi package manager
RUN wget "https://utic-public-cf.s3.amazonaws.com/mesa-gl-24.1.0-r0.718c913d.apk" && \
wget "https://utic-public-cf.s3.amazonaws.com/mesa-glapi-24.1.0-r0.4390a503.apk" && \
apk del mesa-gl && \
apk add --allow-untrusted mesa-gl-24.1.0-r0.718c913d.apk && \
apk add --allow-untrusted mesa-glapi-24.1.0-r0.4390a503.apk && \
rm mesa-gl-24.1.0-r0.718c913d.apk && \
rm mesa-glapi-24.1.0-r0.4390a503.apk
RUN chown -R notebook-user:notebook-user /app && \ RUN chown -R notebook-user:notebook-user /app && \
apk add font-ubuntu git && \ apk add font-ubuntu git && \
fc-cache -fv && \ fc-cache -fv && \

14
requirements/base.txt
View File

@ -4,7 +4,7 @@
# #
# pip-compile ./base.in # pip-compile ./base.in
# #
anyio==4.4.0 anyio==4.5.0
# via httpx # via httpx
backoff==2.2.1 backoff==2.2.1
# via -r ./base.in # via -r ./base.in
@ -16,6 +16,8 @@ certifi==2024.8.30
# httpx # httpx
# requests # requests
# unstructured-client # unstructured-client
cffi==1.17.1
# via cryptography
chardet==5.2.0 chardet==5.2.0
# via -r ./base.in # via -r ./base.in
charset-normalizer==3.3.2 charset-normalizer==3.3.2
@ -26,13 +28,15 @@ click==8.1.7
# via # via
# nltk # nltk
# python-oxmsg # python-oxmsg
cryptography==43.0.1
# via unstructured-client
dataclasses-json==0.6.7 dataclasses-json==0.6.7
# via # via
# -r ./base.in # -r ./base.in
# unstructured-client # unstructured-client
deepdiff==8.0.1 deepdiff==8.0.1
# via unstructured-client # via unstructured-client
emoji==2.12.1 emoji==2.13.0
# via -r ./base.in # via -r ./base.in
exceptiongroup==1.2.2 exceptiongroup==1.2.2
# via anyio # via anyio
@ -82,7 +86,9 @@ packaging==24.1
# unstructured-client # unstructured-client
psutil==6.0.0 psutil==6.0.0
# via -r ./base.in # via -r ./base.in
pypdf==4.3.1 pycparser==2.22
# via cffi
pypdf==5.0.0
# via unstructured-client # via unstructured-client
python-dateutil==2.9.0.post0 python-dateutil==2.9.0.post0
# via unstructured-client # via unstructured-client
@ -133,7 +139,7 @@ typing-inspect==0.9.0
# via # via
# dataclasses-json # dataclasses-json
# unstructured-client # unstructured-client
unstructured-client==0.25.8 unstructured-client==0.25.9
# via -r ./base.in # via -r ./base.in
urllib3==1.26.20 urllib3==1.26.20
# via # via

6
requirements/dev.txt
View File

@ -15,7 +15,7 @@ click==8.1.7
# pip-tools # pip-tools
distlib==0.3.8 distlib==0.3.8
# via virtualenv # via virtualenv
filelock==3.16.0 filelock==3.16.1
# via virtualenv # via virtualenv
identify==2.6.1 identify==2.6.1
# via pre-commit # via pre-commit
@ -32,7 +32,7 @@ packaging==24.1
# build # build
pip-tools==7.4.1 pip-tools==7.4.1
# via -r ./dev.in # via -r ./dev.in
platformdirs==4.3.3 platformdirs==4.3.6
# via # via
# -c ./test.txt # -c ./test.txt
# virtualenv # virtualenv
@ -51,7 +51,7 @@ tomli==2.0.1
# -c ./test.txt # -c ./test.txt
# build # build
# pip-tools # pip-tools
virtualenv==20.26.4 virtualenv==20.26.5
# via pre-commit # via pre-commit
wheel==0.44.0 wheel==0.44.0
# via pip-tools # via pip-tools

4
requirements/extra-paddleocr.txt
View File

@ -4,7 +4,7 @@
# #
# pip-compile ./extra-paddleocr.in # pip-compile ./extra-paddleocr.in
# #
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./base.txt # -c ./base.txt
# httpx # httpx
@ -113,7 +113,7 @@ pillow==10.4.0
# pdf2image # pdf2image
# scikit-image # scikit-image
# unstructured-paddleocr # unstructured-paddleocr
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ././deps/constraints.txt # -c ././deps/constraints.txt
# paddlepaddle # paddlepaddle

26
requirements/extra-pdf-image.txt
View File

@ -13,7 +13,9 @@ certifi==2024.8.30
# -c ./base.txt # -c ./base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./base.txt # -c ./base.txt
@ -24,14 +26,16 @@ coloredlogs==15.0.1
contourpy==1.3.0 contourpy==1.3.0
# via matplotlib # via matplotlib
cryptography==43.0.1 cryptography==43.0.1
# via pdfminer-six # via
# -c ./base.txt
# pdfminer-six
cycler==0.12.1 cycler==0.12.1
# via matplotlib # via matplotlib
deprecated==1.2.14 deprecated==1.2.14
# via pikepdf # via pikepdf
effdet==0.4.1 effdet==0.4.1
# via -r ./extra-pdf-image.in # via -r ./extra-pdf-image.in
filelock==3.16.0 filelock==3.16.1
# via # via
# huggingface-hub # huggingface-hub
# torch # torch
@ -44,9 +48,9 @@ fsspec==2024.9.0
# via # via
# huggingface-hub # huggingface-hub
# torch # torch
google-api-core[grpc]==2.19.2 google-api-core[grpc]==2.20.0
# via google-cloud-vision # via google-cloud-vision
google-auth==2.34.0 google-auth==2.35.0
# via # via
# google-api-core # google-api-core
# google-cloud-vision # google-cloud-vision
@ -63,7 +67,7 @@ grpcio==1.66.1
# grpcio-status # grpcio-status
grpcio-status==1.62.3 grpcio-status==1.62.3
# via google-api-core # via google-api-core
huggingface-hub==0.24.7 huggingface-hub==0.25.0
# via # via
# timm # timm
# tokenizers # tokenizers
@ -166,7 +170,7 @@ proto-plus==1.24.0
# via # via
# google-api-core # google-api-core
# google-cloud-vision # google-cloud-vision
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ././deps/constraints.txt # -c ././deps/constraints.txt
# google-api-core # google-api-core
@ -185,10 +189,12 @@ pyasn1-modules==0.4.1
pycocotools==2.0.8 pycocotools==2.0.8
# via effdet # via effdet
pycparser==2.22 pycparser==2.22
# via cffi # via
# -c ./base.txt
# cffi
pyparsing==3.1.4 pyparsing==3.1.4
# via matplotlib # via matplotlib
pypdf==4.3.1 pypdf==5.0.0
# via # via
# -c ./base.txt # -c ./base.txt
# -r ./extra-pdf-image.in # -r ./extra-pdf-image.in
@ -236,7 +242,7 @@ six==1.16.0
# via # via
# -c ./base.txt # -c ./base.txt
# python-dateutil # python-dateutil
sympy==1.13.2 sympy==1.13.3
# via # via
# onnxruntime # onnxruntime
# torch # torch

6
requirements/huggingface.txt
View File

@ -16,7 +16,7 @@ click==8.1.7
# via # via
# -c ./base.txt # -c ./base.txt
# sacremoses # sacremoses
filelock==3.16.0 filelock==3.16.1
# via # via
# huggingface-hub # huggingface-hub
# torch # torch
@ -25,7 +25,7 @@ fsspec==2024.9.0
# via # via
# huggingface-hub # huggingface-hub
# torch # torch
huggingface-hub==0.24.7 huggingface-hub==0.25.0
# via # via
# tokenizers # tokenizers
# transformers # transformers
@ -82,7 +82,7 @@ six==1.16.0
# via # via
# -c ./base.txt # -c ./base.txt
# langdetect # langdetect
sympy==1.13.2 sympy==1.13.3
# via torch # via torch
tokenizers==0.19.1 tokenizers==0.19.1
# via # via

4
requirements/ingest/airtable.txt
View File

@ -22,9 +22,9 @@ inflection==0.5.1
# via pyairtable # via pyairtable
pyairtable==2.3.3 pyairtable==2.3.3
# via -r ./ingest/airtable.in # via -r ./ingest/airtable.in
pydantic==2.9.1 pydantic==2.9.2
# via pyairtable # via pyairtable
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
requests==2.32.3 requests==2.32.3
# via # via

6
requirements/ingest/astradb.txt
View File

@ -4,7 +4,7 @@
# #
# pip-compile ./ingest/astradb.in # pip-compile ./ingest/astradb.in
# #
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -12,7 +12,7 @@ astrapy==1.4.2
# via -r ./ingest/astradb.in # via -r ./ingest/astradb.in
cassandra-driver==3.29.2 cassandra-driver==3.29.2
# via cassio # via cassio
cassio==0.1.8 cassio==0.1.9
# via astrapy # via astrapy
certifi==2024.8.30 certifi==2024.8.30
# via # via
@ -70,7 +70,7 @@ packaging==24.1
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# deprecation # deprecation
pymongo==4.8.0 pymongo==4.9.1
# via astrapy # via astrapy
requests==2.32.3 requests==2.32.3
# via # via

14
requirements/ingest/azure.txt
View File

@ -23,9 +23,9 @@ azure-core==1.31.0
# azure-storage-blob # azure-storage-blob
azure-datalake-store==0.0.53 azure-datalake-store==0.0.53
# via adlfs # via adlfs
azure-identity==1.17.1 azure-identity==1.18.0
# via adlfs # via adlfs
azure-storage-blob==12.22.0 azure-storage-blob==12.23.0
# via adlfs # via adlfs
certifi==2024.8.30 certifi==2024.8.30
# via # via
@ -33,6 +33,7 @@ certifi==2024.8.30
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via # via
# -c ./ingest/../base.txt
# azure-datalake-store # azure-datalake-store
# cryptography # cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
@ -41,6 +42,7 @@ charset-normalizer==3.3.2
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via # via
# -c ./ingest/../base.txt
# azure-identity # azure-identity
# azure-storage-blob # azure-storage-blob
# msal # msal
@ -74,11 +76,11 @@ multidict==6.1.0
portalocker==2.10.1 portalocker==2.10.1
# via msal-extensions # via msal-extensions
pycparser==2.22 pycparser==2.22
# via cffi
pyjwt[crypto]==2.9.0
# via # via
# msal # -c ./ingest/../base.txt
# pyjwt # cffi
pyjwt[crypto]==2.9.0
# via msal
requests==2.32.3 requests==2.32.3
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt

12
requirements/ingest/box.txt
View File

@ -15,13 +15,17 @@ certifi==2024.8.30
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./ingest/../base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via boxsdk # via
# -c ./ingest/../base.txt
# boxsdk
fsspec==2024.9.0 fsspec==2024.9.0
# via # via
# -r ./ingest/box.in # -r ./ingest/box.in
@ -31,7 +35,9 @@ idna==3.10
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
pycparser==2.22 pycparser==2.22
# via cffi # via
# -c ./ingest/../base.txt
# cffi
pyjwt==2.9.0 pyjwt==2.9.0
# via boxsdk # via boxsdk
python-dateutil==2.9.0.post0 python-dateutil==2.9.0.post0

22
requirements/ingest/chroma.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -45,19 +45,21 @@ coloredlogs==15.0.1
# via onnxruntime # via onnxruntime
deprecated==1.2.14 deprecated==1.2.14
# via opentelemetry-api # via opentelemetry-api
durationpy==0.7
# via kubernetes
exceptiongroup==1.2.2 exceptiongroup==1.2.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# anyio # anyio
fastapi==0.114.2 fastapi==0.115.0
# via chromadb # via chromadb
filelock==3.16.0 filelock==3.16.1
# via huggingface-hub # via huggingface-hub
flatbuffers==24.3.25 flatbuffers==24.3.25
# via onnxruntime # via onnxruntime
fsspec==2024.9.0 fsspec==2024.9.0
# via huggingface-hub # via huggingface-hub
google-auth==2.34.0 google-auth==2.35.0
# via kubernetes # via kubernetes
googleapis-common-protos==1.65.0 googleapis-common-protos==1.65.0
# via opentelemetry-exporter-otlp-proto-grpc # via opentelemetry-exporter-otlp-proto-grpc
@ -73,7 +75,7 @@ h11==0.14.0
# uvicorn # uvicorn
httptools==0.6.1 httptools==0.6.1
# via uvicorn # via uvicorn
huggingface-hub==0.24.7 huggingface-hub==0.25.0
# via tokenizers # via tokenizers
humanfriendly==10.0 humanfriendly==10.0
# via coloredlogs # via coloredlogs
@ -89,7 +91,7 @@ importlib-metadata==8.5.0
# -r ./ingest/chroma.in # -r ./ingest/chroma.in
importlib-resources==6.4.5 importlib-resources==6.4.5
# via chromadb # via chromadb
kubernetes==30.1.0 kubernetes==31.0.0
# via chromadb # via chromadb
monotonic==1.6 monotonic==1.6
# via posthog # via posthog
@ -132,7 +134,7 @@ packaging==24.1
# onnxruntime # onnxruntime
posthog==3.6.6 posthog==3.6.6
# via chromadb # via chromadb
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ./ingest/../deps/constraints.txt # -c ./ingest/../deps/constraints.txt
# googleapis-common-protos # googleapis-common-protos
@ -146,11 +148,11 @@ pyasn1==0.6.1
# rsa # rsa
pyasn1-modules==0.4.1 pyasn1-modules==0.4.1
# via google-auth # via google-auth
pydantic==2.9.1 pydantic==2.9.2
# via # via
# chromadb # chromadb
# fastapi # fastapi
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pypika==0.48.9 pypika==0.48.9
# via chromadb # via chromadb
@ -192,7 +194,7 @@ sniffio==1.3.1
# httpx # httpx
starlette==0.38.5 starlette==0.38.5
# via fastapi # via fastapi
sympy==1.13.2 sympy==1.13.3
# via onnxruntime # via onnxruntime
tenacity==8.5.0 tenacity==8.5.0
# via # via

2
requirements/ingest/clarifai.txt
View File

@ -45,7 +45,7 @@ pillow==10.4.0
# via clarifai # via clarifai
prompt-toolkit==3.0.47 prompt-toolkit==3.0.47
# via inquirerpy # via inquirerpy
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ./ingest/../deps/constraints.txt # -c ./ingest/../deps/constraints.txt
# clarifai-grpc # clarifai-grpc

4
requirements/ingest/databricks-volumes.txt
View File

@ -14,9 +14,9 @@ charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
databricks-sdk==0.32.1 databricks-sdk==0.32.3
# via -r ./ingest/databricks-volumes.in # via -r ./ingest/databricks-volumes.in
google-auth==2.34.0 google-auth==2.35.0
# via databricks-sdk # via databricks-sdk
idna==3.10 idna==3.10
# via # via

12
requirements/ingest/embed-aws-bedrock.txt
View File

@ -14,7 +14,7 @@ aiosignal==1.3.1
# via aiohttp # via aiohttp
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -84,14 +84,14 @@ langchain==0.3.0
# via langchain-community # via langchain-community
langchain-community==0.3.0 langchain-community==0.3.0
# via -r ./ingest/embed-aws-bedrock.in # via -r ./ingest/embed-aws-bedrock.in
langchain-core==0.3.0 langchain-core==0.3.2
# via # via
# langchain # langchain
# langchain-community # langchain-community
# langchain-text-splitters # langchain-text-splitters
langchain-text-splitters==0.3.0 langchain-text-splitters==0.3.0
# via langchain # via langchain
langsmith==0.1.120 langsmith==0.1.125
# via # via
# langchain # langchain
# langchain-community # langchain-community
@ -120,13 +120,13 @@ packaging==24.1
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# langchain-core # langchain-core
# marshmallow # marshmallow
pydantic==2.9.1 pydantic==2.9.2
# via # via
# langchain # langchain
# langchain-core # langchain-core
# langsmith # langsmith
# pydantic-settings # pydantic-settings
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pydantic-settings==2.5.2 pydantic-settings==2.5.2
# via langchain-community # via langchain-community
@ -158,7 +158,7 @@ sniffio==1.3.1
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# anyio # anyio
# httpx # httpx
sqlalchemy==2.0.34 sqlalchemy==2.0.35
# via # via
# langchain # langchain
# langchain-community # langchain-community

19
requirements/ingest/embed-huggingface.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -24,7 +24,7 @@ exceptiongroup==1.2.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# anyio # anyio
filelock==3.16.0 filelock==3.16.1
# via # via
# huggingface-hub # huggingface-hub
# torch # torch
@ -45,7 +45,7 @@ httpx==0.27.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# langsmith # langsmith
huggingface-hub==0.24.7 huggingface-hub==0.25.0
# via # via
# langchain-huggingface # langchain-huggingface
# sentence-transformers # sentence-transformers
@ -67,11 +67,11 @@ jsonpatch==1.33
# via langchain-core # via langchain-core
jsonpointer==3.0.0 jsonpointer==3.0.0
# via jsonpatch # via jsonpatch
langchain-core==0.3.0 langchain-core==0.3.2
# via langchain-huggingface # via langchain-huggingface
langchain-huggingface==0.1.0 langchain-huggingface==0.1.0
# via -r ./ingest/embed-huggingface.in # via -r ./ingest/embed-huggingface.in
langsmith==0.1.120 langsmith==0.1.125
# via langchain-core # via langchain-core
markupsafe==2.1.5 markupsafe==2.1.5
# via jinja2 # via jinja2
@ -84,7 +84,6 @@ numpy==1.26.4
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# scikit-learn # scikit-learn
# scipy # scipy
# sentence-transformers
# transformers # transformers
orjson==3.10.7 orjson==3.10.7
# via langsmith # via langsmith
@ -96,11 +95,11 @@ packaging==24.1
# transformers # transformers
pillow==10.4.0 pillow==10.4.0
# via sentence-transformers # via sentence-transformers
pydantic==2.9.1 pydantic==2.9.2
# via # via
# langchain-core # langchain-core
# langsmith # langsmith
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pyyaml==6.0.2 pyyaml==6.0.2
# via # via
@ -125,14 +124,14 @@ scipy==1.13.1
# via # via
# scikit-learn # scikit-learn
# sentence-transformers # sentence-transformers
sentence-transformers==3.1.0 sentence-transformers==3.1.1
# via langchain-huggingface # via langchain-huggingface
sniffio==1.3.1 sniffio==1.3.1
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# anyio # anyio
# httpx # httpx
sympy==1.13.2 sympy==1.13.3
# via torch # via torch
tenacity==8.5.0 tenacity==8.5.0
# via langchain-core # via langchain-core

6
requirements/ingest/embed-mixedbreadai.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -38,9 +38,9 @@ idna==3.10
# httpx # httpx
mixedbread-ai==2.2.6 mixedbread-ai==2.2.6
# via -r ./ingest/embed-mixedbreadai.in # via -r ./ingest/embed-mixedbreadai.in
pydantic==2.9.1 pydantic==2.9.2
# via mixedbread-ai # via mixedbread-ai
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
sniffio==1.3.1 sniffio==1.3.1
# via # via

8
requirements/ingest/embed-octoai.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -47,11 +47,11 @@ idna==3.10
# requests # requests
jiter==0.5.0 jiter==0.5.0
# via openai # via openai
openai==1.45.0 openai==1.46.1
# via -r ./ingest/embed-octoai.in # via -r ./ingest/embed-octoai.in
pydantic==2.9.1 pydantic==2.9.2
# via openai # via openai
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
regex==2024.9.11 regex==2024.9.11
# via # via

12
requirements/ingest/embed-openai.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -52,13 +52,13 @@ jsonpatch==1.33
# via langchain-core # via langchain-core
jsonpointer==3.0.0 jsonpointer==3.0.0
# via jsonpatch # via jsonpatch
langchain-core==0.3.0 langchain-core==0.3.2
# via langchain-openai # via langchain-openai
langchain-openai==0.2.0 langchain-openai==0.2.0
# via -r ./ingest/embed-openai.in # via -r ./ingest/embed-openai.in
langsmith==0.1.120 langsmith==0.1.125
# via langchain-core # via langchain-core
openai==1.45.0 openai==1.46.1
# via langchain-openai # via langchain-openai
orjson==3.10.7 orjson==3.10.7
# via langsmith # via langsmith
@ -66,12 +66,12 @@ packaging==24.1
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# langchain-core # langchain-core
pydantic==2.9.1 pydantic==2.9.2
# via # via
# langchain-core # langchain-core
# langsmith # langsmith
# openai # openai
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pyyaml==6.0.2 pyyaml==6.0.2
# via langchain-core # via langchain-core

22
requirements/ingest/embed-vertexai.txt
View File

@ -14,7 +14,7 @@ aiosignal==1.3.1
# via aiohttp # via aiohttp
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -50,14 +50,14 @@ frozenlist==1.4.1
# via # via
# aiohttp # aiohttp
# aiosignal # aiosignal
google-api-core[grpc]==2.19.2 google-api-core[grpc]==2.20.0
# via # via
# google-cloud-aiplatform # google-cloud-aiplatform
# google-cloud-bigquery # google-cloud-bigquery
# google-cloud-core # google-cloud-core
# google-cloud-resource-manager # google-cloud-resource-manager
# google-cloud-storage # google-cloud-storage
google-auth==2.34.0 google-auth==2.35.0
# via # via
# google-api-core # google-api-core
# google-cloud-aiplatform # google-cloud-aiplatform
@ -65,7 +65,7 @@ google-auth==2.34.0
# google-cloud-core # google-cloud-core
# google-cloud-resource-manager # google-cloud-resource-manager
# google-cloud-storage # google-cloud-storage
google-cloud-aiplatform==1.66.0 google-cloud-aiplatform==1.67.1
# via langchain-google-vertexai # via langchain-google-vertexai
google-cloud-bigquery==3.25.0 google-cloud-bigquery==3.25.0
# via google-cloud-aiplatform # via google-cloud-aiplatform
@ -135,17 +135,17 @@ langchain==0.3.0
# langchain-community # langchain-community
langchain-community==0.3.0 langchain-community==0.3.0
# via -r ./ingest/embed-vertexai.in # via -r ./ingest/embed-vertexai.in
langchain-core==0.3.0 langchain-core==0.3.2
# via # via
# langchain # langchain
# langchain-community # langchain-community
# langchain-google-vertexai # langchain-google-vertexai
# langchain-text-splitters # langchain-text-splitters
langchain-google-vertexai==2.0.0 langchain-google-vertexai==2.0.1
# via -r ./ingest/embed-vertexai.in # via -r ./ingest/embed-vertexai.in
langchain-text-splitters==0.3.0 langchain-text-splitters==0.3.0
# via langchain # via langchain
langsmith==0.1.120 langsmith==0.1.125
# via # via
# langchain # langchain
# langchain-community # langchain-community
@ -182,7 +182,7 @@ proto-plus==1.24.0
# google-api-core # google-api-core
# google-cloud-aiplatform # google-cloud-aiplatform
# google-cloud-resource-manager # google-cloud-resource-manager
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ./ingest/../deps/constraints.txt # -c ./ingest/../deps/constraints.txt
# google-api-core # google-api-core
@ -198,7 +198,7 @@ pyasn1==0.6.1
# rsa # rsa
pyasn1-modules==0.4.1 pyasn1-modules==0.4.1
# via google-auth # via google-auth
pydantic==2.9.1 pydantic==2.9.2
# via # via
# google-cloud-aiplatform # google-cloud-aiplatform
# langchain # langchain
@ -206,7 +206,7 @@ pydantic==2.9.1
# langchain-google-vertexai # langchain-google-vertexai
# langsmith # langsmith
# pydantic-settings # pydantic-settings
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pydantic-settings==2.5.2 pydantic-settings==2.5.2
# via langchain-community # via langchain-community
@ -243,7 +243,7 @@ sniffio==1.3.1
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# anyio # anyio
# httpx # httpx
sqlalchemy==2.0.34 sqlalchemy==2.0.35
# via # via
# langchain # langchain
# langchain-community # langchain-community

12
requirements/ingest/embed-voyageai.txt
View File

@ -16,7 +16,7 @@ aiosignal==1.3.1
# via aiohttp # via aiohttp
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -69,7 +69,7 @@ jsonpointer==3.0.0
# via jsonpatch # via jsonpatch
langchain==0.3.0 langchain==0.3.0
# via -r ./ingest/embed-voyageai.in # via -r ./ingest/embed-voyageai.in
langchain-core==0.3.0 langchain-core==0.3.2
# via # via
# langchain # langchain
# langchain-text-splitters # langchain-text-splitters
@ -78,7 +78,7 @@ langchain-text-splitters==0.3.0
# via langchain # via langchain
langchain-voyageai==0.1.2 langchain-voyageai==0.1.2
# via -r ./ingest/embed-voyageai.in # via -r ./ingest/embed-voyageai.in
langsmith==0.1.120 langsmith==0.1.125
# via # via
# langchain # langchain
# langchain-core # langchain-core
@ -97,13 +97,13 @@ packaging==24.1
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# langchain-core # langchain-core
pydantic==2.9.1 pydantic==2.9.2
# via # via
# langchain # langchain
# langchain-core # langchain-core
# langchain-voyageai # langchain-voyageai
# langsmith # langsmith
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pyyaml==6.0.2 pyyaml==6.0.2
# via # via
@ -120,7 +120,7 @@ sniffio==1.3.1
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# anyio # anyio
# httpx # httpx
sqlalchemy==2.0.34 sqlalchemy==2.0.35
# via langchain # via langchain
tenacity==8.5.0 tenacity==8.5.0
# via # via

6
requirements/ingest/gcs.txt
View File

@ -42,11 +42,11 @@ fsspec==2024.9.0
# gcsfs # gcsfs
gcsfs==2024.9.0.post1 gcsfs==2024.9.0.post1
# via -r ./ingest/gcs.in # via -r ./ingest/gcs.in
google-api-core==2.19.2 google-api-core==2.20.0
# via # via
# google-cloud-core # google-cloud-core
# google-cloud-storage # google-cloud-storage
google-auth==2.34.0 google-auth==2.35.0
# via # via
# gcsfs # gcsfs
# google-api-core # google-api-core
@ -80,7 +80,7 @@ oauthlib==3.2.2
# via requests-oauthlib # via requests-oauthlib
proto-plus==1.24.0 proto-plus==1.24.0
# via google-api-core # via google-api-core
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ./ingest/../deps/constraints.txt # -c ./ingest/../deps/constraints.txt
# google-api-core # google-api-core

9
requirements/ingest/github.txt
View File

@ -10,6 +10,7 @@ certifi==2024.8.30
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via # via
# -c ./ingest/../base.txt
# cryptography # cryptography
# pynacl # pynacl
charset-normalizer==3.3.2 charset-normalizer==3.3.2
@ -17,7 +18,9 @@ charset-normalizer==3.3.2
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via pyjwt # via
# -c ./ingest/../base.txt
# pyjwt
deprecated==1.2.14 deprecated==1.2.14
# via pygithub # via pygithub
idna==3.10 idna==3.10
@ -25,7 +28,9 @@ idna==3.10
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
pycparser==2.22 pycparser==2.22
# via cffi # via
# -c ./ingest/../base.txt
# cffi
pygithub==2.4.0 pygithub==2.4.0
# via -r ./ingest/github.in # via -r ./ingest/github.in
pyjwt[crypto]==2.9.0 pyjwt[crypto]==2.9.0

8
requirements/ingest/google-drive.txt
View File

@ -14,11 +14,11 @@ charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
google-api-core==2.19.2 google-api-core==2.20.0
# via google-api-python-client # via google-api-python-client
google-api-python-client==2.145.0 google-api-python-client==2.146.0
# via -r ./ingest/google-drive.in # via -r ./ingest/google-drive.in
google-auth==2.34.0 google-auth==2.35.0
# via # via
# google-api-core # google-api-core
# google-api-python-client # google-api-python-client
@ -37,7 +37,7 @@ idna==3.10
# requests # requests
proto-plus==1.24.0 proto-plus==1.24.0
# via google-api-core # via google-api-core
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ./ingest/../deps/constraints.txt # -c ./ingest/../deps/constraints.txt
# google-api-core # google-api-core

2
requirements/ingest/mongodb.txt
View File

@ -6,5 +6,5 @@
# #
dnspython==2.6.1 dnspython==2.6.1
# via pymongo # via pymongo
pymongo==4.8.0 pymongo==4.9.1
# via -r ./ingest/mongodb.in # via -r ./ingest/mongodb.in

2
requirements/ingest/notion.txt
View File

@ -4,7 +4,7 @@
# #
# pip-compile ./ingest/notion.in # pip-compile ./ingest/notion.in
# #
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx

15
requirements/ingest/onedrive.txt
View File

@ -15,13 +15,16 @@ certifi==2024.8.30
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./ingest/../base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via # via
# -c ./ingest/../base.txt
# msal # msal
# pyjwt # pyjwt
idna==3.10 idna==3.10
@ -32,14 +35,14 @@ msal==1.31.0
# via # via
# -r ./ingest/onedrive.in # -r ./ingest/onedrive.in
# office365-rest-python-client # office365-rest-python-client
office365-rest-python-client==2.5.12 office365-rest-python-client==2.5.13
# via -r ./ingest/onedrive.in # via -r ./ingest/onedrive.in
pycparser==2.22 pycparser==2.22
# via cffi
pyjwt[crypto]==2.9.0
# via # via
# msal # -c ./ingest/../base.txt
# pyjwt # cffi
pyjwt[crypto]==2.9.0
# via msal
pytz==2024.2 pytz==2024.2
# via office365-rest-python-client # via office365-rest-python-client
requests==2.32.3 requests==2.32.3

15
requirements/ingest/outlook.txt
View File

@ -9,13 +9,16 @@ certifi==2024.8.30
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./ingest/../base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via # via
# -c ./ingest/../base.txt
# msal # msal
# pyjwt # pyjwt
idna==3.10 idna==3.10
@ -26,14 +29,14 @@ msal==1.31.0
# via # via
# -r ./ingest/outlook.in # -r ./ingest/outlook.in
# office365-rest-python-client # office365-rest-python-client
office365-rest-python-client==2.5.12 office365-rest-python-client==2.5.13
# via -r ./ingest/outlook.in # via -r ./ingest/outlook.in
pycparser==2.22 pycparser==2.22
# via cffi
pyjwt[crypto]==2.9.0
# via # via
# msal # -c ./ingest/../base.txt
# pyjwt # cffi
pyjwt[crypto]==2.9.0
# via msal
pytz==2024.2 pytz==2024.2
# via office365-rest-python-client # via office365-rest-python-client
requests==2.32.3 requests==2.32.3

10
requirements/ingest/qdrant.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# httpx # httpx
@ -55,15 +55,15 @@ numpy==1.26.4
# qdrant-client # qdrant-client
portalocker==2.10.1 portalocker==2.10.1
# via qdrant-client # via qdrant-client
protobuf==4.25.4 protobuf==4.25.5
# via # via
# -c ./ingest/../deps/constraints.txt # -c ./ingest/../deps/constraints.txt
# grpcio-tools # grpcio-tools
pydantic==2.9.1 pydantic==2.9.2
# via qdrant-client # via qdrant-client
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
qdrant-client==1.11.1 qdrant-client==1.11.2
# via -r ./ingest/qdrant.in # via -r ./ingest/qdrant.in
sniffio==1.3.1 sniffio==1.3.1
# via # via

14
requirements/ingest/salesforce.txt
View File

@ -11,13 +11,17 @@ certifi==2024.8.30
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./ingest/../base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via pyjwt # via
# -c ./ingest/../base.txt
# pyjwt
idna==3.10 idna==3.10
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
@ -30,10 +34,12 @@ lxml==5.3.0
# zeep # zeep
more-itertools==10.5.0 more-itertools==10.5.0
# via simple-salesforce # via simple-salesforce
platformdirs==4.3.3 platformdirs==4.3.6
# via zeep # via zeep
pycparser==2.22 pycparser==2.22
# via cffi # via
# -c ./ingest/../base.txt
# cffi
pyjwt[crypto]==2.9.0 pyjwt[crypto]==2.9.0
# via simple-salesforce # via simple-salesforce
pytz==2024.2 pytz==2024.2

9
requirements/ingest/sftp.txt
View File

@ -8,15 +8,20 @@ bcrypt==4.2.0
# via paramiko # via paramiko
cffi==1.17.1 cffi==1.17.1
# via # via
# -c ./ingest/../base.txt
# cryptography # cryptography
# pynacl # pynacl
cryptography==43.0.1 cryptography==43.0.1
# via paramiko # via
# -c ./ingest/../base.txt
# paramiko
fsspec==2024.9.0 fsspec==2024.9.0
# via -r ./ingest/sftp.in # via -r ./ingest/sftp.in
paramiko==3.5.0 paramiko==3.5.0
# via -r ./ingest/sftp.in # via -r ./ingest/sftp.in
pycparser==2.22 pycparser==2.22
# via cffi # via
# -c ./ingest/../base.txt
# cffi
pynacl==1.5.0 pynacl==1.5.0
# via paramiko # via paramiko

15
requirements/ingest/sharepoint.txt
View File

@ -9,13 +9,16 @@ certifi==2024.8.30
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./ingest/../base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via # via
# -c ./ingest/../base.txt
# msal # msal
# pyjwt # pyjwt
idna==3.10 idna==3.10
@ -26,14 +29,14 @@ msal==1.31.0
# via # via
# -r ./ingest/sharepoint.in # -r ./ingest/sharepoint.in
# office365-rest-python-client # office365-rest-python-client
office365-rest-python-client==2.5.12 office365-rest-python-client==2.5.13
# via -r ./ingest/sharepoint.in # via -r ./ingest/sharepoint.in
pycparser==2.22 pycparser==2.22
# via cffi
pyjwt[crypto]==2.9.0
# via # via
# msal # -c ./ingest/../base.txt
# pyjwt # cffi
pyjwt[crypto]==2.9.0
# via msal
pytz==2024.2 pytz==2024.2
# via office365-rest-python-client # via office365-rest-python-client
requests==2.32.3 requests==2.32.3

2
requirements/ingest/slack.txt
View File

@ -4,5 +4,5 @@
# #
# pip-compile ./ingest/slack.in # pip-compile ./ingest/slack.in
# #
slack-sdk==3.32.0 slack-sdk==3.33.1
# via -r ./ingest/slack.in # via -r ./ingest/slack.in

12
requirements/ingest/weaviate.txt
View File

@ -11,19 +11,25 @@ certifi==2024.8.30
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cffi==1.17.1 cffi==1.17.1
# via cryptography # via
# -c ./ingest/../base.txt
# cryptography
charset-normalizer==3.3.2 charset-normalizer==3.3.2
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
cryptography==43.0.1 cryptography==43.0.1
# via authlib # via
# -c ./ingest/../base.txt
# authlib
idna==3.10 idna==3.10
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt
# requests # requests
pycparser==2.22 pycparser==2.22
# via cffi # via
# -c ./ingest/../base.txt
# cffi
requests==2.32.3 requests==2.32.3
# via # via
# -c ./ingest/../base.txt # -c ./ingest/../base.txt

8
requirements/test.txt
View File

@ -6,7 +6,7 @@
# #
annotated-types==0.7.0 annotated-types==0.7.0
# via pydantic # via pydantic
anyio==4.4.0 anyio==4.5.0
# via # via
# -c ./base.txt # -c ./base.txt
# httpx # httpx
@ -121,7 +121,7 @@ pathspec==0.12.1
# via black # via black
pillow==10.4.0 pillow==10.4.0
# via label-studio-sdk # via label-studio-sdk
platformdirs==4.3.3 platformdirs==4.3.6
# via black # via black
pluggy==1.5.0 pluggy==1.5.0
# via pytest # via pytest
@ -129,11 +129,11 @@ pycodestyle==2.12.1
# via # via
# flake8 # flake8
# flake8-print # flake8-print
pydantic==2.9.1 pydantic==2.9.2
# via # via
# -r ./test.in # -r ./test.in
# label-studio-sdk # label-studio-sdk
pydantic-core==2.23.3 pydantic-core==2.23.4
# via pydantic # via pydantic
pyflakes==3.2.0 pyflakes==3.2.0
# via # via

2
unstructured/__version__.py
View File

@ -1 +1 @@
__version__ = "0.15.13-dev3" # pragma: no cover __version__ = "0.15.13" # pragma: no cover