mirror of
https://github.com/Unstructured-IO/unstructured.git
synced 2025-08-15 12:19:36 +00:00
5376bc510f
* add python-magic * first pass on filetype detection * tests for filetype detection * more tests for file detection * added tests for error conditions * install libmagic dev in github * libmagic install instructions * pattern for checking email files * support reading .eml in rb mode * add auto partition function * auto tests for emal * auto tests for docx * added tests for html * add pdf and html tests * linting, linting, linting * added docs for auto partitioning * update readme with generic partition brick * bumped version * added test for bad type * detect .docx files from application/octet-stream * linting, linting, linting * identify xlsx from octet stream * install poppler in ci * fix mocks; test for unknown type * install poppler utils * install in one line * only poppler-utils * file extension logic from application/octet-stream * install local inference for ci * install detectron2 * removing unused dockerfile
Example Docs
The sample docs directory contains the following files:
example-10k.html- A 10-K SEC filing in HTML formatlayout-parser-paper.pdf- A PDF copy of the layout parser paperfactbook.xml/factbook.xsl- Example XML/XLS files that you can use to test stylesheets
These documents can be used to test out the parsers in the library. In addition, here are instructions for pullig in some sample docs that are too big to store in the repo.
XBRL 10-K
You can get an example 10-K in inline XBRL format using the following curl. Note, you need
to have the user agent set in the header or the SEC site will reject your request.
curl -O \
-A '${organization} ${email}'
https://www.sec.gov/Archives/edgar/data/311094/000117184321001344/0001171843-21-001344.txt
You can parse this document using the HTML parser.