llama-hub/loader_hub/file/README.md

# File Loader

This loader takes in a local directory containing files and extracts `Document`s from each of the files.

## Usage

To use this loader, you simply need to instantiate the `SimpleDirectoryReader` class with a directory, along with other optional settings, such as whether to ignore hidden files. See the code for the complete list.

```python
from loader_hub import SimpleDirectoryReader

loader = SimpleDirectoryReader('data', recursive=True, exclude_hidden=True)
documents = loader.load_data()
```

## Examples

This loader is designed to be used as a way to load data into [GPT Index](https://github.com/jerryjliu/gpt_index/tree/main/gpt_index) and/or subsequently used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent.

### GPT Index

```python
from loader_hub import SimpleDirectoryReader
from gpt_index import GPTSimpleVectorIndex

loader = SimpleDirectoryReader('data', recursive=True, exclude_hidden=True)
documents = loader.load_data()
index = GPTSimpleVectorIndex(documents)
index.query('What are these files about?')
```

### LangChain

Note: Make sure you change the description of the `Tool` to match your use-case.

```python
from loader_hub import SimpleDirectoryReader
from gpt_index import GPTSimpleVectorIndex
from langchain.agents import initialize_agent, Tool
from langchain.llms import OpenAI
from langchain.chains.conversation.memory import ConversationBufferMemory

loader = SimpleDirectoryReader('data', recursive=True, exclude_hidden=True)
documents = loader.load_data()
index = GPTSimpleVectorIndex(documents)

tools = [
    Tool(
        name="Local Directory Index",
        func=lambda q: index.query(q),
        description=f"Useful when you want answer questions about the files in your local directory.",
    ),
]
llm = OpenAI(temperature=0)
memory = ConversationBufferMemory(memory_key="chat_history")
agent_chain = initialize_agent(
    tools, llm, agent="zero-shot-react-description", memory=memory
)

output = agent_chain.run(input="What are these files about?")
```