1 Commits

Author SHA1 Message Date
ahmetkca
5a27264db1
Add GitHub Repository Reader (#34)
* add github repository, test a new way to download loader

* test imports when downloaded from gpt_index

* Refactor(Github Repo): Move github_client and utils to modules

* Moved github_client.py and utils.py from loader_hub/github_repo to modules/github_repo
* Updated import statements in base.py to reflect the new location

* temp

* Refactor(GithubRepositoryReader): Add github_client argument

- Add github_client argument to GithubRepositoryReader constructor
- Set default value for github_client argument
- Update docstring to reflect changes

* Refactor(Github Repo): Update init file

- Remove imports of base, github_client and utils
- Add imports of GithubRepositoryReader and GithubClient
- Update __all__ to include the new imports

* Fix(library): Update library.json

- Updated library.json to include __init__.py file

* Refactor(GithubRepositoryReader): Add filter for directories and files

- Add filter for directories and files in GithubRepositoryReader
- Ignore directories and files that do not pass the filter
- Print out if directory or file is ignored due to filter

* Refactor(BaseReader): Check filter files

- Refactor `_check_filter_files` to `_check_filter_file_extensions` in `BaseReader`
- Ignoring files due to filter

* Docs(FilterType): Add documentation for FilterType enum

- Add documentation for FilterType enum
- Explain what the enum is used for
- Describe the attributes of the enum

* Add(GPT Index): Add GPT Index example

Add GPT Index example to README
- Set OPENAI_API_KEY environment variable
- Download GithubRepositoryReader module
- Create GithubClient and GithubRepositoryReader
- Load data from Github Repository
- Create GPTSimpleVectorIndex
- Query the index

* Add(GPT Index): Add GPT Index example

Add GPT Index example to README
- Set OPENAI_API_KEY environment variable
- Download GithubRepositoryReader module
- Create GithubClient and GithubRepositoryReader
- Load data from Github Repository
- Create GPTSimpleVectorIndex
- Query the index

* Add(GPT Index): Add GPT Index example

Add GPT Index example to README
- Set OPENAI_API_KEY environment variable
- Download GithubRepositoryReader module
- Create GithubClient and GithubRepositoryReader
- Load data from Github Repository
- Create GPTSimpleVectorIndex
- Query the index

* change the import path for extras

* change import path for extra files to absolute

* Add test for GithubClient currently not using mocks which is not ideal

* Update test_github_reader.py

* Update test_github_reader.py

---------

Co-authored-by: Jesse Zhang <jessetanzhang@gmail.com>
2023-02-24 23:41:48 -08:00