mirror of
https://github.com/deepset-ai/haystack.git
synced 2026-02-07 07:22:03 +00:00
69 lines
2.4 KiB
Plaintext
69 lines
2.4 KiB
Plaintext
---
|
|
title: "CSVToDocument"
|
|
id: csvtodocument
|
|
slug: "/csvtodocument"
|
|
description: "Converts CSV files to documents."
|
|
---
|
|
|
|
# CSVToDocument
|
|
|
|
Converts CSV files to documents.
|
|
|
|
<div className="key-value-table">
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
|
| **Mandatory run variables** | `sources`: A list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects |
|
|
| **Output variables** | `documents`: A list of documents |
|
|
| **API reference** | [Converters](/reference/converters-api) |
|
|
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/csv.py |
|
|
|
|
</div>
|
|
|
|
## Overview
|
|
|
|
`CSVToDocument` converts one or more CSV files into a text document.
|
|
|
|
The component uses UTF-8 encoding by default, but you may specify a different encoding if needed during initialization.
|
|
You can optionally attach metadata to each document with a `meta` parameter when running the component.
|
|
|
|
## Usage
|
|
|
|
### On its own
|
|
|
|
```python
|
|
from haystack.components.converters.csv import CSVToDocument
|
|
|
|
converter = CSVToDocument()
|
|
results = converter.run(sources=["sample.csv"], meta={"date_added": datetime.now().isoformat()})
|
|
documents = results["documents"]
|
|
|
|
print(documents[0].content)
|
|
## 'col1,col2\now1,row1\nrow2row2\n'
|
|
```
|
|
|
|
### In a pipeline
|
|
|
|
```python
|
|
from haystack import Pipeline
|
|
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
|
from haystack.components.converters import CSVToDocument
|
|
from haystack.components.preprocessors import DocumentCleaner
|
|
from haystack.components.preprocessors import DocumentSplitter
|
|
from haystack.components.writers import DocumentWriter
|
|
|
|
document_store = InMemoryDocumentStore()
|
|
|
|
pipeline = Pipeline()
|
|
pipeline.add_component("converter", CSVToDocument())
|
|
pipeline.add_component("cleaner", DocumentCleaner())
|
|
pipeline.add_component("splitter", DocumentSplitter(split_by="sentence", split_length=5))
|
|
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
|
pipeline.connect("converter", "cleaner")
|
|
pipeline.connect("cleaner", "splitter")
|
|
pipeline.connect("splitter", "writer")
|
|
|
|
pipeline.run({"converter": {"sources": file_names}})
|
|
```
|