yujunjun/unstructured
1
0
Fork 0
mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-07-08 17:46:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
908 Commits 211 Branches 192 Tags
Commit Graph

68 Commits

Author SHA1 Message Date
Matt Robinson
1ce8447ba7
build(deps): bump unstructured inference; compile from setup.py (#176) 
* bump unstructured inference; compile from setup.py

* bump version

* compile the local-inference extra

* linting, linting, linting
 2023-01-25 16:32:57 +00:00
Matt Robinson
59f972d739
build(deps): add requests as a base dependency (#162) 
* build(deps): add `requests` as a base dependency

* linting, linting, linting

* changelog typo
 2023-01-18 16:36:23 +00:00
Matt Robinson
419c0867d3
build(deps): bump unstructured_inference version range (#151) 
* bump unstructured-inference to 0.2.3

* bump version
 2023-01-13 22:21:36 +00:00
Matt Robinson
eba4c80b1e
feat: get_directory_file_info for exploring a directory of files (#142) 
* added python-pptx to requirements

* added filetype detection for powerpoint

* add more filetypes to detect

* more tests

* added tests for filetype

* reorder document types

* tests for get_directory_file_info

* added docs for get_directory_file_info

* bump version

* Word -> Office

* added test for filetype

* add group by filetype example
 2023-01-11 12:40:50 -05:00
Matt Robinson
5376bc510f
feat: generic partition brick with filetype detection (#132) 
* add python-magic

* first pass on filetype detection

* tests for filetype detection

* more tests for file detection

* added tests for error conditions

* install libmagic dev in github

* libmagic install instructions

* pattern for checking email files

* support reading .eml in rb mode

* add auto partition function

* auto tests for emal

* auto tests for docx

* added tests for html

* add pdf and html tests

* linting, linting, linting

* added docs for auto partitioning

* update readme with generic partition brick

* bumped version

* added test for bad type

* detect .docx files from application/octet-stream

* linting, linting, linting

* identify xlsx from octet stream

* install poppler in ci

* fix mocks; test for unknown type

* install poppler utils

* install in one line

* only poppler-utils

* file extension logic from application/octet-stream

* install local inference for ci

* install detectron2

* removing unused dockerfile
 2023-01-09 16:15:14 -05:00
qued
a75499d465
feat: local inference (#125) 
Splits partition_pdf into two paths, one used for local inference when url is None, another for inference via api when url is a string.
 2023-01-04 16:19:05 -06:00
Matt Robinson
17045aed80
feat: add convert_to_dataframe staging brick (#127) 
* add pandas to deps; pip-compile

* staging brick to convert elements to dataframe

* bump version

* add convert_to_dataframe docs

* bump wheel version

* typo fix

* typo fix 2!
 2023-01-04 12:04:59 -05:00
Matt Robinson
b14f6ac9bd
feat: extract metadata from .docx, .xlsx, and .jpg (#113) 
* add python-docx dependency

* added function for extracting metadata from word documents

* add openpyxl

* added get_jpg_metadata; fixed typing

* bump changelog

* added pillow to dependencies
 2022-12-26 09:34:36 -05:00
Matt Robinson
407f700b20
build(deps): bump certify to incorporate security patches (#105) 
* pin certifi in base and huggingface

* pinning for build and docs
 2022-12-19 14:47:15 -05:00
Matt Robinson
b1cce16c16
feat: translate_text cleaning brick (#101) 
* initial implementation for translate brick

* more input validation

* tests for translate brick

* added docs

* bumped version

* chinese and arabic tests

* re-run pip-compile

* add torch to dependencies

* cleanup doc string

* fix long string

* fix typo in docs

* take out empty string check

* return string if string is empty

* added huggingface into make install
 2022-12-15 15:35:15 -05:00
Matt Robinson
5c4428413a
build(deps): Bump jupyter-core library (#85) 2022-11-30 10:04:56 -05:00
asymness
2170a2aae2
feat: Implement Argilla staging brick (#81) 
* Add argilla to dependencies and run pip-compile

* Implement Argilla staging brick and add unit tests

* Update version and changelog

* Update docs with description and usage for Argilla staging brick

* Remove unused fixtures and fix typo in Argilla tests

* add missing quote in docs

* changelog tweak

* doc tweaks

Co-authored-by: Matt Robinson <mrobinson@unstructuredai.io>
Co-authored-by: Matt Robinson <mrobinson@unstructured.io>
 2022-11-28 14:41:48 +00:00
Mallori Harrell
53fcf4e912
chore: Remove PDF parsing code and dependencies (#75) 
Remove PDF parsing code and dependencies.
 2022-11-21 11:47:29 -06:00
Yuming Long
7c61639f23
python_require (#65) 2022-11-11 12:15:23 -05:00
Matt Robinson
2715950d6f
chore: Add long description content type; bump version (#59) 2022-11-08 16:55:41 -05:00
benjats07
6b3e86c508
docs: Added long description to PyPi (#58) 
* docs: Added long description to PyPi

* Added fields for description in PyPi
 2022-11-08 15:22:43 -06:00
Matt Robinson
fb16847946
feat: Staging brick for attention window chunking (#34) 
* add huggingface dependencies and re pip-compile

* first pass on chunk by attention window

* test for chunking function

* completed tests for chunk_by_attention_window

* change default buffer size to 2

* wrapper function for staging

* added docs for transformers

* fix wording and typos

* updated change log and bumped the version

* added docs on huggingface dependencies

* fix typo

* re pip-compile
 2022-10-13 11:18:27 -04:00
Matt Robinson
5f40c78f25 Initial Release 2022-09-26 14:55:20 -07:00