yujunjun/llama-hub
1
0
Fork 0
mirror of https://github.com/run-llama/llama-hub.git synced 2025-08-13 11:11:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
llama-hub/loader_hub/file/cjk_pdf
History
Jerry Liu e97bb81915
swap out gpt_index imports for llama_index imports (#49) 
* cr

* cr

* cr

---------

Co-authored-by: Jerry Liu <jerry@robustintelligence.com>
Co-authored-by: Jesse Zhang <jessetanzhang@gmail.com>
2023-02-20 21:46:58 -08:00
..
__init__.py
Add Japanese PDF reader (#35)
2023-02-15 09:03:03 -08:00
base.py
swap out gpt_index imports for llama_index imports (#49)
2023-02-20 21:46:58 -08:00
README.md
swap out gpt_index imports for llama_index imports (#49)
2023-02-20 21:46:58 -08:00
requirements.txt
Add Japanese PDF reader (#35)
2023-02-15 09:03:03 -08:00

README.md

Chinese/Japanese/Korean PDF Loader

This loader extracts the text from a local PDF file using the pdfminer.six Python package, which is used instead of PyPDF2 in order to load Asian languages, e.g. shift-jis encoded Japanese text. The officially supported characters are those in CJK (Chinese, Japanese, and Korean), though it may work for other languages as well. Any non-text elements are ignored. A single local file is passed in each time you call load_data.

Usage

To use this loader, you need to pass in a Path to a local file.

from pathlib import Path
from llama_index import download_loader

CJKPDFReader = download_loader("CJKPDFReader")

loader = CJKPDFReader()
documents = loader.load_data(file=Path('./article.pdf'))

This loader is designed to be used as a way to load data into LlamaIndex and/or subsequently used as a Tool in a LangChain Agent. See here for examples.