unstructured/requirements/test.in

-c ./deps/constraints.txt
-c base.txt
black>=22.3.0
coverage
types-click
flake8
flake8-print
freezegun
label_studio_sdk
mypy
pydantic
pytest-cov
pytest-mock
ruff
types-Markdown
types-requests
types-tabulate
vcrpy
grpcio
autoflake
liccheck
feat: update dependencies and remove constraint on pydantic (#2841) ### Description * The `consistent-deps.sh` was fixed to take into account the ingest dependencies, causing some errors to show up. New constriants were added to make that script pass. * Update all requirements without constraint on pydantic, allowing the latest version to be pulled in. * `pikepdf` is causing a conflict but there's a fix on their `main` branch, just need for the next release to be published. Opened up a question here to see if we can get that out any sooner: [Do releases happen on a schedule?](https://github.com/pikepdf/pikepdf/discussions/574). For now added `lxml<5` to the constraints. A couple optimizations: * `constraints.in` renamed to `constraints.txt` since the whole point is all dependencies are already pinned and the file never gets compiled * `constraints.txt` moved to a `requirements/deps` directory as this never gets compiled by `pip-compile` * Other dependency files updated to reference the new location of `base.in` and `constraints.txt` * make file updated since it was originally written to avoid the `base.in` and `constraints.in` file 2024-04-04 15:58:23 -04:00			`-c ./deps/constraints.txt`
build(deps): avoid version conflicts (#636) Addresses #631. * Uses constraints to keep dependency versions more consistent. * Moves all dependencies to .in files which are then ingested by setup.py. * Adds script to check consistency of all extras. * Adds consistency check to CI. I should note that while it shouldn't be possible to cause a conflict between base.txt and any of the extras (because base.txt constrains all the extras) it is possible to get a conflict between two of the extras files. There are ways of trying to avoid that (like constraining each file by all the files that have already been processed before it in the order given in the make pip-compile target) but the ones I could think of seemed a little overwrought, and come with problems of their own. If a conflict arises, it should be flagged by CI or locally with make check-deps. When/if that happens, you can resolve the conflict by adding appropriate global constraints in requirements/constraints.txt. Also note that if fileA.in is constrained by fileB.txt, then fileB.in should be compiled before fileA.in in the make pip-compile target. Otherwise fileA.in will be compiled with the old version of fileB.txt which can cause conflicts or keep dependencies from being updated properly. 2023-05-24 17:29:35 -05:00			`-c base.txt`
Initial Release 2022-06-29 14:35:19 -04:00			`black>=22.3.0`
			`coverage`
Roman/ingest refactor (#978) * Pull out s3 code as subcommand * Pull out dropbox code as subcommand * Pull out azure code as subcommand * Pull out fsspec code as subcommand * Pull out github code as subcommand * Pull out gitlab code as subcommand * Pull out reddit code as subcommand * Pull out slack code as subcommand * Pull out discord code as subcommand * Pull out wikipedia code as subcommand * Pull out gdrive code as subcommand * Pull out biomed code as subcommand * rename parameters * Pull out onedrive code as subcommand * Pull out outlook code as subcommand * Pull out local code as subcommand * Pull out elasticsearch code as subcommand * Pull out confluence code as subcommand * Drop previous main file * update changelog * Add back in mp.Pool * Fix mypy issues with click * Make sure all tests run with verbose flag * refactor approach to dynamically add common options to each subcommand, scrub logging of options for sensitive data * Pull out some more shared options * Support running code via python as well as cli * update ingest readme and move it to the ingest folder * update usage in connector docs * move local command arg in test * Seperate out cli code from logic running unstructured * Make some cli fields required rather than optional * rename process -> processor * Improve logger to avoid duplicate handlers --------- Co-authored-by: Ryan Nikolaidis <1208590+ryannikolaidis@users.noreply.github.com> 2023-07-31 13:20:10 -04:00			`types-click`
Initial Release 2022-06-29 14:35:19 -04:00			`flake8`
chore: lint for print statements in ingest code (#2215) ### Description Given the filtering in the ingest logger, anything going to console should go through that. This adds a linter that only checks for `print()` statements in the ingest code and ignored it elsewhere for now. 2023-12-05 11:42:23 -05:00			`flake8-print`
feat: adds data_source metadata to ElementMetadata (#690) 2023-06-07 21:22:18 -07:00			`freezegun`
			`label_studio_sdk`
Initial Release 2022-06-29 14:35:19 -04:00			`mypy`
chore: Bump inference and set default ocr_mode to entire_page (#1172) * pip-compile in order to bump unstructured-inference * Set the default `ocr_mode` back to `enitre_page` now that [this error](https://github.com/Unstructured-IO/unstructured-inference/pull/183) is addressed * Explicitly add `sphinx-tabs` to `build.in`. This file provides `docs/requirements.txt`. * Remove a pinned `pydantic` version * Fix a makefile command to `pip-compile` a missing ingest file. 2023-08-22 19:05:02 -04:00			`pydantic`
Initial Release 2022-06-29 14:35:19 -04:00			`pytest-cov`
feat: add ability to pass headers in partition_html (#397) Also adds pytest-mock requirement, those fixtures are nice to have! Implements issue/feature #396 . 2023-03-23 20:14:57 -07:00			`pytest-mock`
build: remove ruff version upper bound (#3829) Summary Remove pin on `ruff` linter and fix the handful of lint errors a newer version catches. 2024-12-16 15:01:22 -08:00			`ruff`
feat: adds data_source metadata to ElementMetadata (#690) 2023-06-07 21:22:18 -07:00			`types-Markdown`
feat: `partition_html` directly from a url (#202) * added tests for html from url * bump version * added types-requests * and -> an 2023-02-07 09:09:34 -05:00			`types-requests`
feat: adds data_source metadata to ElementMetadata (#690) 2023-06-07 21:22:18 -07:00			`types-tabulate`
chore: Integration test to show LabelStudio brick working with SDK (#21) 2022-10-05 14:38:44 -04:00			`vcrpy`
Chore: Allow passing kwargs to request data field (#716) * bump again :( * update to kwarg * add test case * rename to request_kwargs * remove install detectron2 * pip compile * add changelog for remove detectron2 install * resolve weaviate import issue on python 3.9 2023-06-12 12:39:58 -04:00			`grpcio`
refactoring linting (#1739) ### Description Currently linting only takes place over the base unstructured directory but we support python files throughout the repo. It makes sense for all those files to also abide by the same linting rules so the entire repo was set to be inspected when the linters are run. Along with that autoflake was added as a linter which has a lot of added benefits such as removing unused imports for you that would currently break flake and require manual intervention. The only real relevant changes in this PR are in the `Makefile`, `setup.cfg`, and `requirements/test.in`. The rest is the result of running the linters. 2023-10-17 08:45:12 -04:00			`autoflake`
build: check dependency licenses in CI (#3349) ### Summary Adds a CI check to ensure that packages added as dependencies are appropriately licensed. All of the `.txt` files in the `requirements` directory are checked with the exception of: - `constraints.txt`, since those are not installed and are instead conditions on the other dependency files - `dev.txt`, since those are for local development and not shipped as part of the `unstructured` package - `extra-pdf-image.txt` - the `extra-pdf-image.in` since checking `extra-pdf-image.txt` pulls in NVIDIA GPU related packages with an `Other/Proprietary` license type, and there's not a good way to exclude those without adding `Other/Proprietary` to the allowed licenses list. ### Testing The new `check-licenses` job should pass in CI. 2024-07-11 18:36:01 -04:00			`liccheck`