mirror of
https://github.com/Unstructured-IO/unstructured.git
synced 2025-06-27 02:30:08 +00:00
c82bad1061
Addresses #631. * Uses constraints to keep dependency versions more consistent. * Moves all dependencies to .in files which are then ingested by setup.py. * Adds script to check consistency of all extras. * Adds consistency check to CI. I should note that while it shouldn't be possible to cause a conflict between base.txt and any of the extras (because base.txt constrains all the extras) it is possible to get a conflict between two of the extras files. There are ways of trying to avoid that (like constraining each file by all the files that have already been processed before it in the order given in the make pip-compile target) but the ones I could think of seemed a little overwrought, and come with problems of their own. If a conflict arises, it should be flagged by CI or locally with make check-deps. When/if that happens, you can resolve the conflict by adding appropriate global constraints in requirements/constraints.txt. Also note that if fileA.in is constrained by fileB.txt, then fileB.in should be compiled before fileA.in in the make pip-compile target. Otherwise fileA.in will be compiled with the old version of fileB.txt which can cause conflicts or keep dependencies from being updated properly.
46 lines
1.3 KiB
Bash
Executable File
46 lines
1.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
####################################################################################################
|
|
# Check depedency consistency by forcing pip to resolve all the requirement .txt files at once
|
|
# (without installing).
|
|
####################################################################################################
|
|
|
|
echo "Checking consistency of dependencies..."
|
|
|
|
# Joins an array of strings using the specified delimiter.
|
|
function join_by {
|
|
local d=${1-} f=${2-}
|
|
if shift 2; then
|
|
printf %s "$f" "${@/#/$d}"
|
|
fi
|
|
}
|
|
|
|
# NOTE(alan): Add any dependency files here we don't want to include in the resolution.
|
|
excludefiles=("requirements/build.txt")
|
|
|
|
# Build an array of requirements files.
|
|
shopt -s nullglob
|
|
reqfiles=(requirements/*.txt)
|
|
|
|
# Remove the excluded files from the array of requirements files.
|
|
for excludefile in "${excludefiles[@]}"; do
|
|
for i in "${!reqfiles[@]}"; do
|
|
if [[ ${reqfiles[i]} = "$excludefile" ]]; then
|
|
unset 'reqfiles[i]'
|
|
fi
|
|
done
|
|
done
|
|
|
|
# Turn the requirement files array into pip -r flags.
|
|
reqstring=$(join_by ' -r ' "${reqfiles[@]}")
|
|
reqstring="-r ${reqstring}"
|
|
|
|
# This pip command will attempt to resolve the dependencies without installing anything.
|
|
pipcommand="pip install --dry-run --ignore-installed ${reqstring}"
|
|
if $pipcommand >> /dev/null;
|
|
then
|
|
echo "Everything looks fine!";
|
|
else
|
|
exit 1
|
|
fi
|