28 Commits

Author SHA1 Message Date
Massimiliano Pippi
3728a95de6
fix docker tag for cuda (#2952) 2022-08-03 17:59:46 +02:00
Malte Pietsch
4bf470286b
Upgrade xpdf to 4.0.4 (#2443)
* Update minimal gpu docker image to xpdf 4.0.4

* Update Dockerfile-GPU

* Update Dockerfile

* Update Dockerfile-GPU

* Update Dockerfile-GPU-minimal
2022-04-21 10:27:56 +02:00
Julian Risch
ac5617e757
Add basic telemetry features (#2314)
* add basic telemetry features

* change pipeline_config to _component_config

* Update Documentation & Code Style

* add super().__init__() calls to error classes

* make posthog mock work with python 3.7

* Update Documentation & Code Style

* update link to docs web page

* log exceptions, send event for raised HaystackErrors, refactor Path(CONFIG_PATH)

* add comment on send_event in BaseComponent.init() and fix mypy

* mock NonPrivateParameters and fix pylint undefined-variable

* Update Documentation & Code Style

* check model path contains multiple /

* add test for writing to file

* add test for en-/disable telemetry

* Update Documentation & Code Style

* merge file deletion methods and ignore pylint global statement

* Update Documentation & Code Style

* set env variable in demo to activate telemetry

* fix mock of HAYSTACK_TELEMETRY_ENABLED

* fix mypy and linter

* add CI as env variable to execution contexts

* remove threading, add test for custom error event

* Update Documentation & Code Style

* simplify config/log file deletion

* add test for final event being sent

* force writing config file in test

* make test compatible with python 3.7

* switch to posthog production server

* Update Documentation & Code Style

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-21 11:58:51 +01:00
Sara Zan
9dc89d2bd2
Fix dependency related build issues in Dockerfiles (#2135)
* Fix a path issue in Dockerfile-GPU

* Fix paths in Dockerfile-GPU

* Add workflow_dispatch to docker build task

* Remove reference to optional component from ui/, not needed anymore

* Move pytorch installation last to avoid replacing it later

* Remove optional import from rest_api too, no more needed

* Change path in ui/Dockerfile

* ui container works again

* Complete review of import paths

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-09 17:35:18 +01:00
Sara Zan
692cde11e7
Change docstores_gpu into docstores-gpu in Dockerfile-GPU (#2129) 2022-02-07 15:10:17 +01:00
Sara Zan
c6bfb1c1d4
Remove rest_api extra frpom Dockerfile-GPU (#2122) 2022-02-04 16:06:40 +01:00
Sara Zan
d470b9d0bd
Improve dependency management (#1994)
* Fist attempt at using setup.cfg for dependency management

* Trying the new package on the CI and in Docker too

* Add composite extras_require

* Add the safe_import function for document store imports and add some try-catch statements on rest_api and ui imports

* Fix bug on class import and rephrase error message

* Introduce typing for optional modules and add type: ignore in sparse.py

* Include importlib_metadata backport for py3.7

* Add colab group to extra_requires

* Fix pillow version

* Fix grpcio

* Separate out the crawler as another extra

* Make paths relative in rest_api and ui

* Update the test matrix in the CI

* Add try catch statements around the optional imports too to account for direct imports

* Never mix direct deps with self-references and add ES deps to the base install

* Refactor several paths in tests to make them insensitive to the execution path

* Include tstadel review and re-introduce Milvus1 in the tests suite, to fix

* Wrap pdf conversion utils into safe_import

* Update some tutorials and rever Milvus1 as default for now, see #2067

* Fix mypy config


Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-01-26 18:12:55 +01:00
oryx1729
cb881b6fa9
Disable pip cache for Dockerfiles (#2015) 2022-01-19 10:26:17 +01:00
Sara Zan
9c3d9b4885
Add models to demo docker image (#1978)
* Add utility to cache models and nltk data & modify Dockerfiles to use it

* Fix punkt data not being cached
2022-01-11 16:37:45 +01:00
oryx1729
5b3f693562
Fix Dockerfile-GPU (#1969) 2022-01-06 11:13:04 +01:00
oryx1729
e075663feb
Upgrade torch version (#1960) 2022-01-05 18:14:14 +01:00
oryx1729
00c823cdff
Add GitHub Action for Docker Build for GPU (#1916) 2022-01-04 14:33:13 +01:00
Timo Moeller
6892955e95
Add execute permissions (#1666) 2021-10-27 17:35:34 +02:00
Timo Moeller
6da2c73611
Add nltk download, add folder for file upload (#1633) 2021-10-22 16:03:33 +02:00
Malte Pietsch
bb9ec90d3c
Fix tesseract installation in Dockerfile (#1405)
* Fix Dockerfile

* Update Dockerfile-GPU
2021-09-02 11:09:30 +02:00
Alvise Sembenico
6326cf5710
🐳 add PDF converter dependencies to Docker (#1107) 2021-05-31 19:01:02 +02:00
oryx1729
6d00eff796
Add PDF converter in Dockerfiles (#877) 2021-03-08 09:55:11 +01:00
Malte Pietsch
d700592c9a
Update GPU Dockerimage (Cuda 11, Fix faiss)(#836) 2021-02-17 12:40:00 +01:00
Malte Pietsch
74b0868d28
Fix GPU docker build (#703) 2020-12-31 15:04:13 +01:00
Malte Pietsch
46530e86f8
Fix sentencepiece dependency in dockerfiles (#553) 2020-11-05 12:01:27 +01:00
Guillim
7a43d1a72d
Update readme path in Dockerfile (#537)
* Update Dockerfile

forgot to change the extension i believe

* Update Dockerfile

* Update Dockerfile-GPU
2020-11-03 10:19:18 +01:00
Malte Pietsch
a92ca04648
Update GPU docker & fix race condition with multiple workers (#436)
* fix gpu CMD and set tag to latest

* udpate dockerfiles. resolve race condition of index creation with multiple workers

* update dockerfiles for preload. remove try catch for elastic index creation

* add back try/catch. disable multiproc in default config to comply with --preload of gunicorn

* change to pip3 for GPU dockerfile

* remove --preload for gpu
2020-09-29 21:12:44 +02:00
Malte Pietsch
4da480aa15 Fix dockerfiles 2020-07-16 15:58:49 +02:00
Guillim
8a616dae75
Adjust Docker and REST API to allow TransformsReader Class (#180) 2020-07-07 16:25:36 +02:00
Tanay Soni
dcb5be45b2
Update Python version in Dockerfile-GPU (#71) 2020-04-22 19:41:21 +02:00
Tanay Soni
51a3851f93
Update Dockerfiles to use Gunicorn for deployment (#69) 2020-04-21 16:14:51 +02:00
Malte Pietsch
76c5c1d6aa
Improve deployment of REST API (Configs, logging, minor bugs) (#40)
* remove env variables from dockerfiles

* add more config options to rest api. make fields optional. change to elasticsearch as default

* skip reader if retriever doesn't return anything

* add more config params to farm reader. fix top_k_per_sample

* update FARM version
2020-03-18 12:26:13 +01:00
Malte Pietsch
2164e8550f
Add gpu dockerfile, improve logging, fix minor bug with filtering (#36)
* add gpu dockerfile. improve logging. fix minor bug with filtering

* fix path
2020-03-12 18:30:42 +01:00