yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-09-22 14:43:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

build: do not install 'dev' extras with 'all' (#4888)

Browse Source 
* do not install 'dev' with 'all'

* some fixes around
This commit is contained in:
Massimiliano Pippi 2023-05-11 19:24:47 +02:00 committed by GitHub
parent 618699eb52
commit d322beed6c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 35 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/examples-tests.yml vendored
View File

@ -40,7 +40,7 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }} python-version: ${{ env.PYTHON_VERSION }}
- name: Install Haystack - name: Install Haystack
run: pip install .[all] run: pip install .[all,dev]
- name: Run - name: Run
run: pytest examples/ run: pytest examples/

4
.github/workflows/linting.yml vendored
View File

@ -36,7 +36,7 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }} python-version: ${{ env.PYTHON_VERSION }}
- name: Install Haystack - name: Install Haystack
run: pip install ".[all]" run: pip install ".[all,dev]"
- name: Mypy - name: Mypy
if: steps.files.outputs.any_changed == 'true' if: steps.files.outputs.any_changed == 'true'
@ -70,7 +70,7 @@ jobs:
- name: Install Haystack - name: Install Haystack
run: | run: |
pip install ".[all]" pip install ".[all,dev]"
pip install ./haystack-linter pip install ./haystack-linter
- name: Pylint - name: Pylint

2
.github/workflows/openapi_sync.yml vendored
View File

@ -26,7 +26,7 @@ jobs:
- name: Install Haystack - name: Install Haystack
run: | run: |
pip install --upgrade pip pip install --upgrade pip
pip install -U -e .[all] pip install -U -e .[all,dev]
pip install -e ./rest_api pip install -e ./rest_api
- name: Update OpenAPI specs - name: Update OpenAPI specs

10
.github/workflows/tests.yml vendored
View File

@ -155,7 +155,7 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }} python-version: ${{ env.PYTHON_VERSION }}
- name: Install Haystack - name: Install Haystack
run: pip install .[all] run: pip install .[all,dev]
- name: Run - name: Run
run: pytest --cov-report xml:coverage.xml --cov="haystack" -m "unit" test/${{ matrix.topic }} run: pytest --cov-report xml:coverage.xml --cov="haystack" -m "unit" test/${{ matrix.topic }}
@ -1051,7 +1051,7 @@ jobs:
sudo apt-get install ffmpeg sudo apt-get install ffmpeg
- name: Install Haystack - name: Install Haystack
run: pip install .[all] run: pip install .[all,dev]
- name: Run tests - name: Run tests
env: env:
@ -1138,7 +1138,7 @@ jobs:
# - name: Install sndfile (audio support) # https://github.com/libsndfile/libsndfile/releases/download/1.1.0/libsndfile-1.1.0-win64.zip # - name: Install sndfile (audio support) # https://github.com/libsndfile/libsndfile/releases/download/1.1.0/libsndfile-1.1.0-win64.zip
- name: Install Haystack - name: Install Haystack
run: pip install .[all] run: pip install .[all,dev]
- name: Run tests - name: Run tests
env: env:
@ -1252,7 +1252,7 @@ jobs:
sudo apt-get install libsndfile1 ffmpeg sudo apt-get install libsndfile1 ffmpeg
- name: Install Haystack - name: Install Haystack
run: pip install .[all] run: pip install .[all,dev]
- name: Cache HF models - name: Cache HF models
id: cache-hf-models id: cache-hf-models
@ -1371,7 +1371,7 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }} python-version: ${{ env.PYTHON_VERSION }}
- name: Install Haystack - name: Install Haystack
run: pip install .[all] run: pip install .[all,dev]
- name: Run tests - name: Run tests
env: env:

2
CONTRIBUTING.md
View File

@ -54,7 +54,7 @@ pip install --upgrade pip
# Install Haystack in editable mode # Install Haystack in editable mode
pip install -e '.[dev]' pip install -e '.[dev]'
``` ```
Note that the `.[dev]` part is enough in many development scenarios when adding minor code fixes. However, if your changes require a schema change, then you'll need to install all dependencies with `pip install -e '.[all]' ` command. Introducing new components or changing their interface requires a schema change. Note that the `.[dev]` part is enough in many development scenarios when adding minor code fixes. However, if your changes require a schema change, then you'll need to install all dependencies with `pip install -e '.[all,dev]' ` command. Introducing new components or changing their interface requires a schema change.
This will install all the dependencies you need to work on the codebase, plus testing and formatting dependencies. This will install all the dependencies you need to work on the codebase, plus testing and formatting dependencies.
Last, install the pre-commit hooks with: Last, install the pre-commit hooks with:

44
README.md
View File

@ -71,19 +71,19 @@ An example pipeline would consist of one `Retriever` Node and one `Reader` Node.
- **Continuous Learning**: Collect new training data from user feedback in production & improve your models continuously. - **Continuous Learning**: Collect new training data from user feedback in production & improve your models continuously.
## Resources ## Resources
| | | | | |
| --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 📒 [Docs](https://docs.haystack.deepset.ai) | Components, Pipeline Nodes, Guides, API Reference | | 📒 [Docs](https://docs.haystack.deepset.ai) | Components, Pipeline Nodes, Guides, API Reference |
| 💾 [Installation](https://github.com/deepset-ai/haystack#-installation) | How to install Haystack | | 💾 [Installation](https://github.com/deepset-ai/haystack#-installation) | How to install Haystack |
| 🎓 [Tutorials](https://haystack.deepset.ai/tutorials) | See what Haystack can do with our Notebooks & Scripts | | 🎓 [Tutorials](https://haystack.deepset.ai/tutorials) | See what Haystack can do with our Notebooks & Scripts |
| 🎉 [Haystack Extras](https://github.com/deepset-ai/haystack-extras) | A repository that lists extra Haystack packages and components that can be installed separately. | | 🎉 [Haystack Extras](https://github.com/deepset-ai/haystack-extras) | A repository that lists extra Haystack packages and components that can be installed separately. |
| 🔰 [Demos](https://github.com/deepset-ai/haystack-demos) | A repository containing Haystack demo applications with Docker Compose and a REST API | | 🔰 [Demos](https://github.com/deepset-ai/haystack-demos) | A repository containing Haystack demo applications with Docker Compose and a REST API |
| 🖖 [Community](https://github.com/deepset-ai/haystack#-community) | [Discord](https://haystack.deepset.ai/community/join), [Twitter](https://twitter.com/deepset_ai), [Stack Overflow](https://stackoverflow.com/questions/tagged/haystack), [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) | | 🖖 [Community](https://github.com/deepset-ai/haystack#-community) | [Discord](https://haystack.deepset.ai/community/join), [Twitter](https://twitter.com/deepset_ai), [Stack Overflow](https://stackoverflow.com/questions/tagged/haystack), [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) |
| 💙 [Contributing](https://github.com/deepset-ai/haystack#-contributing) | We welcome all contributions! | | 💙 [Contributing](https://github.com/deepset-ai/haystack#-contributing) | We welcome all contributions! |
| 📊 [Benchmarks](https://haystack.deepset.ai/benchmarks/) | Speed & Accuracy of Retriever, Readers and DocumentStores | | 📊 [Benchmarks](https://haystack.deepset.ai/benchmarks/) | Speed & Accuracy of Retriever, Readers and DocumentStores |
| 🔭 [Roadmap](https://haystack.deepset.ai/overview/roadmap) | Public roadmap of Haystack | | 🔭 [Roadmap](https://haystack.deepset.ai/overview/roadmap) | Public roadmap of Haystack |
| 📰 [Blog](https://haystack.deepset.ai/blog) | Learn about the latest with Haystack and NLP | | 📰 [Blog](https://haystack.deepset.ai/blog) | Learn about the latest with Haystack and NLP |
| ☎️ [Jobs](https://www.deepset.ai/jobs) | We're hiring! Have a look at our open positions | | ☎️ [Jobs](https://www.deepset.ai/jobs) | We're hiring! Have a look at our open positions |
## 💾 Installation ## 💾 Installation
@ -94,25 +94,25 @@ For a detailed installation guide see [the official documentation](https://docs.
Use [pip](https://github.com/pypa/pip) to install a basic version of Haystack's latest release: Use [pip](https://github.com/pypa/pip) to install a basic version of Haystack's latest release:
``` ```sh
pip install farm-haystack pip install farm-haystack
``` ```
This command installs everything needed for basic Pipelines that use an Elasticsearch DocumentStore. This command installs everything needed for basic Pipelines that use an in-memory DocumentStore.
**Full Installation** **Full Installation**
To use more advanced features, like certain DocumentStores, FileConverters, OCR, or Ray, install further dependencies. The following command installs the [latest release](https://github.com/deepset-ai/haystack/releases) of Haystack and all its dependencies: To use more advanced features, like certain DocumentStores, FileConverters, OCR, or Ray,
you need to install further dependencies. The following command installs the [latest release](https://github.com/deepset-ai/haystack/releases) of Haystack and all its dependencies:
```bash ```bash
pip install --upgrade pip
pip install 'farm-haystack[all]' ## or 'all-gpu' for the GPU-enabled dependencies pip install 'farm-haystack[all]' ## or 'all-gpu' for the GPU-enabled dependencies
``` ```
If you want to try out the newest features that are not in an official release yet, you can install Haystack from the main branch. The following command installs from `main` branch with `dev` dependencies: If you want to try out the newest features that are not in an official release yet, you can install the unstable version from the main branch with the following command:
``` ```sh
pip install git+https://github.com/deepset-ai/haystack.git@main#egg=farm-haystack[dev] pip install git+https://github.com/deepset-ai/haystack.git@main#egg=farm-haystack
``` ```
To be able to make changes to Haystack code, install with the following commands: To be able to make changes to Haystack code, install with the following commands:
@ -127,8 +127,8 @@ cd haystack
# Upgrade pip # Upgrade pip
pip install --upgrade pip pip install --upgrade pip
# Install Haystack in editable mode # Install Haystack in editable mode (-e) with the development tools
pip install -e '.[all]' pip install -e '.[dev]'
``` ```
If you want to contribute to the Haystack repo, check our [Contributor Guidelines](#💙-contributing) first. If you want to contribute to the Haystack repo, check our [Contributor Guidelines](#💙-contributing) first.

4
pyproject.toml
View File

@ -227,11 +227,11 @@ formatting = [
] ]
all = [ all = [
"farm-haystack[docstores,audio,crawler,preprocessing,file-conversion,pdf,ocr,ray,dev,onnx,beir,metrics]", "farm-haystack[docstores,audio,crawler,preprocessing,file-conversion,pdf,ocr,ray,onnx,beir,metrics]",
] ]
all-gpu = [ all-gpu = [
# beir is incompatible with faiss-gpu: https://github.com/beir-cellar/beir/issues/71 # beir is incompatible with faiss-gpu: https://github.com/beir-cellar/beir/issues/71
"farm-haystack[docstores-gpu,audio,crawler,preprocessing,file-conversion,pdf,ocr,ray,dev,onnx-gpu,metrics]", "farm-haystack[docstores-gpu,audio,crawler,preprocessing,file-conversion,pdf,ocr,ray,onnx-gpu,metrics]",
] ]
[project.scripts] [project.scripts]