yujunjun/llama-hub
1
0
Fork 0
mirror of https://github.com/run-llama/llama-hub.git synced 2025-08-13 03:01:46 +00:00
Code Issues Packages Projects Releases Wiki Activity
llama-hub/loader_hub/remote
History
Jesse Zhang 88df41a27c
Update base.py (#219)
2023-04-27 17:57:27 -07:00
..
__init__.py
Remote reader (#17)
2023-02-09 17:27:20 -08:00
base.py
Update base.py (#219)
2023-04-27 17:57:27 -07:00
README.md
swap out gpt_index imports for llama_index imports (#49)
2023-02-20 21:46:58 -08:00

README.md

Remote Page/File Loader

This loader makes it easy to extract the text from any remote page or file using just its url. If there's a file at the url, this loader will download it temporarily and parse it using SimpleDirectoryReader. It is an all-in-one tool for (almost) any url.

As a result, any page or type of file is supported. For instance, if a .txt url such as a Project Gutenberg book is passed in, the text will be parsed as is. On the other hand, if a hosted .mp3 url is passed in, it will be downloaded and parsed using AudioTranscriber.

Usage

To use this loader, you need to pass in a Path to a local file. Optionally, you may specify a file_extractor for the SimpleDirectoryReader to use, other than the default one.

from llama_index import download_loader

RemoteReader = download_loader("RemoteReader")

loader = RemoteReader()
documents = loader.load_data(url="https://en.wikipedia.org/wiki/File:Example.jpg")

This loader is designed to be used as a way to load data into LlamaIndex and/or subsequently used as a Tool in a LangChain Agent. See here for examples.