MarkusOdenthal 65a1f5e6eb
Fix Notion Loader: For a Database we're only loading the first 100 page_ids (#68)
* Fix Notion Loader: Database the loader is only loading the first 100 page_ids

During testing of the Notion Loader today, I noticed that I only received the first 100 pages when loading all pages from a database. Notion API returns the attribute has_more, and if this is true, we must request more pages. So this is the implementation of the fix.

* Implement suggestions from @emptycrown

* Update base.py

---------

Co-authored-by: Markus Odenthal <markus.odenthal@real-digital.de>
Co-authored-by: Jesse Zhang <jessetanzhang@gmail.com>
2023-02-27 11:18:30 -08:00
..
2023-02-03 00:05:28 -08:00

Notion Loader

This loader loads documents from Notion. The user specifies an API token to initialize the NotionPageReader. They then specify a set of page_ids or database_id to load in the corresponding Document objects.

Usage

Here's an example usage of the NotionPageReader.

from llama_index import download_loader
import os

NotionPageReader = download_loader('NotionPageReader')

integration_token = os.getenv("NOTION_INTEGRATION_TOKEN")
page_ids = ["<page_id>"]
reader = NotionPageReader(integration_token=integration_token)
documents = reader.load_data(page_ids=page_ids)

This loader is designed to be used as a way to load data into LlamaIndex and/or subsequently used as a Tool in a LangChain Agent. See here for examples.