yujunjun/llama-hub
1
0
Fork 0
mirror of https://github.com/run-llama/llama-hub.git synced 2025-08-13 19:21:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
llama-hub/loader_hub/web/async_web
History

README.md

Async Website Loader

This loader is an asynchronous web scraper that fetches the text from static websites by converting the HTML to text.

Usage

To use this loader, you need to pass in an array of URLs.

from llama_index import download_loader

AsyncWebPageReader = download_loader("AsyncWebPageReader")

loader = AsyncWebPageReader()
documents = loader.load_data(urls=['https://google.com'])