mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-12-31 00:57:37 +00:00
f13443054a
* Adding dummy generator implementation * Adding tutorial to try the model * Committing current non working code * Committing current update where we need to call generate function directly and need to convert embedding to tensor way * Addressing review comments. * Refactoring finder, and implementing rag_generator class. * Refined the implementation of RAGGenerator and now it is in clean shape * Renaming RAGGenerator to RAGenerator * Reverting change from finder.py and addressing review comments * Remove support for RagSequenceForGeneration * Utilizing embed_passage function from DensePassageRetriever * Adding sample test data to verify generator output * Updating testing script * Updating testing script * Fixing bug related to top_k * Updating latest farm dependency * Comment out farm dependency * Reverting changes from TransformersReader * Adding transformers dataset to compare transformers and haystack generator implementation * Using generator_encoder instead of question_encoder to generate context_input_ids * Adding workaround to install FARM dependency from master branch * Removing unnecessary changes * Fixing generator test * Removing transformers datasets * Fixing generator test * Some cleanup and updating TODO comments * Adding tutorial notebook * Updating tutorials with comments * Explicitly passing token model in RAG test * Addressing review comments * Fixing notebook * Refactoring tests to reduce memory footprint * Split generator tests in separate ci step and before running it reclaim memory by terminating containers * Moving tika dependent test to separate dir * Remove unwanted code * Brining reader under session scope * Farm is now session object hence restoring changes from default value * Updating assert for pdf converter * Dummy commit to trigger CI flow * REducing memory footprint required for generator tests * Fixing mypy issues * Marking test with tika and elasticsearch markers. Reverting changes in CI and pytest splits * reducing changes * Fixing CI * changing elastic search ci * Fixing test error * Disabling return of embedding * Marking generator test as well * Refactoring tutorials * Increasing ES memory to 750M * Trying another fix for ES CI * Reverting CI changes * Splitting tests in CI * Generator and non-generator markers split * Adding pytest.ini to add markers and enable strict-markers option * Reducing elastic search container memory * Simplifying generator test by using documents with embedding directly * Bump up farm to 0.5.0
55 lines
1.3 KiB
YAML
55 lines
1.3 KiB
YAML
name: Build
|
|
|
|
on:
|
|
push:
|
|
branches: [ master ]
|
|
pull_request:
|
|
branches: [ master ]
|
|
|
|
jobs:
|
|
build:
|
|
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v2
|
|
|
|
- name: Configure sysctl limits for Elasticsearch
|
|
run: |
|
|
sudo swapoff -a
|
|
sudo sysctl -w vm.swappiness=1
|
|
sudo sysctl -w fs.file-max=262144
|
|
sudo sysctl -w vm.max_map_count=262144
|
|
|
|
- name: Run Elasticsearch
|
|
run: docker run -d -p 9200:9200 -e "discovery.type=single-node" -e 'ES_JAVA_OPTS=-Xms500m -Xmx500m' elasticsearch:7.9.2
|
|
|
|
- name: Run Apache Tika
|
|
run: docker run -d -p 9998:9998 apache/tika:1.24.1
|
|
|
|
- name: Set up Python 3.7
|
|
uses: actions/setup-python@v2
|
|
with:
|
|
python-version: 3.7
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
pip install pytest
|
|
pip install -r requirements.txt
|
|
pip install -e .
|
|
|
|
- name: Run Pytest without generator marker
|
|
run: cd test && pytest -m "not generator"
|
|
|
|
- name: Stop Containers
|
|
run: docker rm -f `docker ps -a -q`
|
|
|
|
- name: Run pytest with generator marker
|
|
run: cd test && pytest -m generator
|
|
|
|
- name: Test with mypy
|
|
run: |
|
|
pip install mypy
|
|
mypy haystack --ignore-missing-imports
|