109 Commits

Author	SHA1	Message	Date
Matt Robinson	74e6b84b41	feat: add metadata tracking to document elements (#225) ... * add metadata field to elements * metadata tracking for pdf/image * metadata for html * update expected outputs * metadata for the rest of the document types * take out file metadata for now * add url to tables * added metadata to test_auto * bump version * added coordinates to __init__ * fix coordinates in tests	2023-02-15 18:26:20 +00:00
Matt Robinson	e73cf09977	feat: optional page breaks for .pptx, .pdf, .html and images (#205) ... * page breaks for pptx * added page breaks for image/pdf * tests for images with page breaks * page breaks for html documents * linting, linting, linting * changelog and bump version * update docs * fix typo * refactor reusable code to common.py * add type back in	2023-02-08 15:11:15 +00:00
Matt Robinson	e6cfde5c4a	fix: no UserWarning when partition_pdf is called (#179)	2023-01-27 12:08:18 -05:00
Matt Robinson	339c133326	fix: cleanup from live .docx tests (#177) ... * add env var for cap threshold; raise default threshold * update docs and tests * added check for ending in a comma * update docs * no caps check for all upper text * capture Text in html and text * check category in Text equality check * lower case all caps before checking for verbs * added check for us city/state/zip * added address type * add address to html * add address to text * fix for text tests; escape for large text segments * refactor regex for readability * update comment * additional test for text with linebreaks * update docs * update changelog * update elements docs * remove old comment * case -> cast * type fix	2023-01-26 15:52:25 +00:00
Matt Robinson	8b6c5fac9d	feat: basic PowerPoint parsing in partition_pptx (#166) ... * parition pptx and tests * add parition_pptx to auto * update doc types in readme * add pptx docs * bump version * remove extra whitespace * partition -> partitioning	2023-01-23 17:03:09 +00:00
Matt Robinson	c1822911a5	chore: return Element objects in partition_pdf and partition_image (#164) ... * helper function to convert to element * test for element types * fix for healthcheck url * version bump * note on coordinates * mention FigureCaption * test_shared -> test_common * add check boxes for checkbox template * update changelog	2023-01-19 14:29:28 +00:00
qued	8abf1f119d	feat: partition image (#144) ... Adds partition_image to partition image file types, which is integrated into the partition brick. This relies on the 0.2.2 version of unstructured-inference.	2023-01-13 22:24:13 -06:00
Matt Robinson	f12240c5e7	feat: add support for .txt files in partition (#150) ... * added partition_text for auto * rename partition_text tests * bump version and update docs	2023-01-13 16:39:53 -05:00
Matt Robinson	5376bc510f	feat: generic partition brick with filetype detection (#132) ... * add python-magic * first pass on filetype detection * tests for filetype detection * more tests for file detection * added tests for error conditions * install libmagic dev in github * libmagic install instructions * pattern for checking email files * support reading .eml in rb mode * add auto partition function * auto tests for emal * auto tests for docx * added tests for html * add pdf and html tests * linting, linting, linting * added docs for auto partitioning * update readme with generic partition brick * bumped version * added test for bad type * detect .docx files from application/octet-stream * linting, linting, linting * identify xlsx from octet stream * install poppler in ci * fix mocks; test for unknown type * install poppler utils * install in one line * only poppler-utils * file extension logic from application/octet-stream * install local inference for ci * install detectron2 * removing unused dockerfile	2023-01-09 16:15:14 -05:00