yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-07-24 01:10:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
844 Commits 73 Branches 243 Tags
Commit Graph

39 Commits

Author SHA1 Message Date
Julian Risch
f9d2f786ca
Replace FARM import statements; add dependencies (#1492) 
* Replace FARM import statements; add dependencies

* Add InferenceProc., TextCl.Proc., TextPairCl.Proc.

* Remove FARMRanker, add type annotations, rename max_sample

* Add sample_to_features_text for InferenceProc.

* Fix type annotations: model_name_or_path is str not Path

* Fix mypy errors: implement _create_dataset in TextCl.Proc.

* Add task_type "embeddings" in Inferencer

* Allow loading AdaptiveModel for embedding task

* Add SQuAD eval metrics; enable InferenceProc for embedding task

* Add baskets as param to log_samples and handle empty basket list in log_samples

* Remove unused dependencies

* Remove FARMClassifier (doc classificer) due to ref to TextClassificationHead

* Remove FARMRanker and Classifier from doc generation scripts

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2021-09-28 16:34:24 +02:00
bogdankostic
c644e2b4d0
Add comment to tutorial notebooks about restarting runtime in colab (#1486) 
* Add comment to tutorial notebooks about restarting runtime in colab

* Add latest docstring and tutorial changes

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2021-09-23 14:36:20 +02:00
Julian Risch
d569e66bc7
Update Tutorial1_Basic_QA_Pipeline.ipynb (#1489) 
* Update Tutorial1_Basic_QA_Pipeline.ipynb

passing params to pipeline as dict

* Add latest docstring and tutorial changes

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2021-09-22 16:35:20 +02:00
Branden Chan
2c4baa7f4e
Regenerate API and Tutorial md files (#1480) 
* Change punctuation

* Add latest docstring and tutorial changes

* Change punctuation

* Add documentation for Docs2Answer

* Add latest docstring and tutorial changes

* Generate new API docs

* Replace Finder with Pipeline

* Add latest docstring and tutorial changes

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2021-09-21 14:42:18 +02:00
oryx1729
9dd7c74f4f
Refactor communication between Pipeline Components (#1321) 2021-09-10 11:41:16 +02:00
Malte Pietsch
be9d19afa5
Remove Finder from tutorials (#1329) 2021-08-10 11:50:59 +02:00
Branden Chan
783893c3d2
Tutorial update (#1166) 
* Add header / footer

* Add Milvus example

* Generate md files

* Fix mypy CI
 2021-06-11 11:09:15 +02:00
Julian Risch
a7ba146246
Removed comma from last item in json list (#1114) 2021-06-01 12:32:21 +02:00
Julian Risch
40ceaf418a
Fixing grpcio-tools to version of colab's pre-installed grpcio (#1113) 2021-05-31 19:09:10 +02:00
Malte Pietsch
e91518ee00
Update tutorials (torch versions, ES version, replace Finder with Pipeline) (#814) 
* remove manual torch install on colab

* update elasticsearch version everywhere to 7.9.2

* fix FAQPipeline

* update tutorials with new pipelines

* Add latest docstring and tutorial changes

* revert faqpipeline change. fix field names in tutorial 4

* Add latest docstring and tutorial changes

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2021-02-09 14:56:54 +01:00
Branden Chan
7376185b65
Create DPR training tutorial (#708) 
* WIP: Start DPR training tutorial

* Create basics of DPR Train tutorial

* Update documentation

* Allow DPR to be initialized without document store

* WIP: Add param descriptions to DPR notebook

* Clean tutorial

* Improve loading

* Make doc store optional when loading DPR

* Satisfy mypy type check

* Add links

* Add tutorial header

* Add colab badge

* Clear outputs

* Incorporate reviewer feedback

* WIP: Start DPR training tutorial

* Create basics of DPR Train tutorial

* Update documentation

* Allow DPR to be initialized without document store

* WIP: Add param descriptions to DPR notebook

* Clean tutorial

* Improve loading

* Make doc store optional when loading DPR

* Satisfy mypy type check

* Add links

* Add tutorial header

* Add colab badge

* Clear outputs

* Incorporate reviewer feedback

* Add readme links

* Regenerate tutorials

* Add excitement

* Fix typo

* Fix hard negatives comment

* Wrap tutorial for windows users

* Fix mypy issue
 2021-01-13 10:33:55 +01:00
Malte Pietsch
94b7345505
Make use_gpu=True the default in tutorials (#692) 
* enable gpu args in tutorials

* add info box for gpu runtime on colab
 2020-12-22 07:58:12 +01:00
Branden Chan
e72f4f4299
Update Colab Torch Version (#576) 
* Update torch version

* Update torch version
 2020-11-11 13:55:10 +01:00
bogdankostic
f62117c232
Add urllib version requirement to colab notebooks (#509) 2020-10-23 10:43:58 +02:00
Lalit Pagaria
63c12371b9
Change arg "model" to "model_name_or_path" in TransformersReader (#510) 
* Consistent parameter naming for TransformersReader along with removing unused imports as well.

* Addressing review comments
 2020-10-21 17:15:35 +02:00
Malte Pietsch
747e0c0046
Bump FARM to 0.4.9. Remove custom torch installation from colab tutorials (#404) 2020-09-21 10:26:12 +02:00
Branden Chan
7fdb85d63a
Create documentation website (#272) 
* Skeleton of doc website

* Flesh out documentation pages

* Split concepts into their own rst files

* add tutorial rsts

* Consistent level 1 markdown headers in tutorials

* Change theme to readthedocs

* Turn bullet points into prose

* Populate sections

* Add more text

* Add more sphinx files

* Add more retriever documentation

* combined all documenations in one structure

* rename of src to _src as it was ignored by git

* Incorporate MP2's changes

* add benchmark bar charts

* Adapt docstrings in Readers

* Improvements to intro, creation of glossary

* Adapt docstrings in Retrievers

* Adapt docstrings in Finder

* Adapt Docstrings of Finder

* Updates to text

* Edit text

* update doc strings

* proof read tutorials

* Edit text

* Edit text

* Add stacked chart

* populate graph with data

* Switch Documentation to markdown (#386)

* add way to generate markdown files to sphinx

* changed from rst to markdown and extended sphinx for it

* fix spelling

* Clean titles

* delete file

* change spelling

* add sections to document store usage

* add basic rest api docs

* fix readme in setup.py

* Update Tutorials

* Change section names

* add windows note to pip install

* update intro

* new renderer for markdown files

* Fix typos

* delete dpr_utils.py

* fix windows note in get started

* Fix docstrings

* deleted rest api docs in api

* fixed typo

* Fix docstring

* revert readme to rst

* Fix readme

* Update setup.py

Co-authored-by: deepset <deepset@Crenolape.localdomain>
Co-authored-by: PiffPaffM <markuspaff.mp@gmail.com>
Co-authored-by: Bogdan Kostić <bogdankostic@web.de>
Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
 2020-09-18 12:57:32 +02:00
Malte Pietsch
9727829cc6
Rename and restructure modules (database, indexing, schemas) (#379) 
* rename database to documentstore

* move document, label, multilabel to haystack/schema.py

* rename documentstore -> document_store

* split indexing modules -> file_converter + preprocessor

* fix order of imports

* Update tutorial notebooks

* fix torch version in tutorial 4
 2020-09-16 18:33:23 +02:00
Malte Pietsch
bde33ddaaa
Bump FARM version to 0.4.8 and PyTorch >=1.5.1, <= 1.6.0 (#376) 
* bump farm version to 0.4.8

* move back to original transformers pipeline

* remove dpr_utils and use transformers implementation

* update tutorial notebooks
 2020-09-16 17:24:40 +02:00
Branden Chan
a54d6a5bd7
Make Tutorials Work on Colab GPUs (#322) 
* Add pip install torch+cu
 2020-08-19 14:52:50 +02:00
Malte Pietsch
fe33a481ad
Update tutorials (#200) 
* fix link in readme. update installation in tutorials

* update haystack version to latest master

* add basic documentation for input to write_documents()

* add docstring for sqldocumentstore

* comment out docker in notebook
 2020-07-07 14:59:01 +02:00
Malte Pietsch
07ecfb60b9
Dense Passage Retriever (Inference) (#167) 2020-06-30 19:05:45 +02:00
Tanay Soni
71e15a5a11
Update Haystack version in tutorials (#136) 2020-06-08 11:31:12 +02:00
Tanay Soni
ef9e4f4467
Add PDF text extraction (#109) 2020-06-08 11:07:19 +02:00
Branden Chan
cbe62044b1
Update colab link 2020-05-26 11:56:24 +02:00
Malte Pietsch
d595886630 split docs into passages in tutorials 2020-04-30 19:27:15 +02:00
Malte Pietsch
7b01fb3fbc Merge branch 'master' of github.com:deepset-ai/haystack 2020-04-30 19:03:44 +02:00
Malte Pietsch
7972038afc update tutorials 2020-04-30 19:00:41 +02:00
Malte Pietsch
438543a18a
pin haystack version in tutorials until release (#87) 2020-04-30 18:44:44 +02:00
Tanay Soni
887bdcc376
Update tutorials to use Elasticsearch, new Retrievers (#79) 2020-04-29 14:01:05 +02:00
bogdankostic
0048ee9c5c
Added Jupyter notebooks of Tutorials (#43) 
Add Jupyter and Colab notebooks of tutorials
 2020-03-17 19:58:53 +01:00
Tanay Soni
f83a164095
Add Elasticsearch Document Store (#13) 2020-01-24 18:24:07 +01:00
Tanay Soni
c52266e520 Update tutorials (#12) 
Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
 2020-01-23 15:18:41 +01:00
Malte Pietsch
cab0932fab
Refactor pipeline for better generalizability & Add TransformersReader (#1) 
* add flag to skip writing docs to non-empty db

* change finder pipeline structure for better generalizability

* add basic TransformersReader

* update tutorials and requirements
 2020-01-13 18:56:22 +01:00
Tanay Soni
6bc228fa6a Fetch QA model from remote in tutorial notebook 2019-11-28 12:07:04 +01:00
Malte Pietsch
3367b46348 switch name from farm_haystack to haystack 2019-11-27 13:56:03 +01:00
Malte Pietsch
5f8d9bc7b8 update tutorial. del example.py. rename adaptive_model module 2019-11-25 17:41:03 +01:00
Malte Pietsch
515d65e7c4 update tutorial 2019-11-25 16:14:37 +01:00
Malte Pietsch
d32fd12aed Move and update tutorial. Improve logging 2019-11-25 16:01:32 +01:00