* remove unused columns and change property document_attribute_columns to metadata
* format file
* fix 'metadata' column on output
* run check
* fix test on nltk
* remove docs changes
* add text_splitting unit test
* change folder test text splitting
* fix chunk fn
* test new function
* run formatter
* run spell check
* run semver
* remove tiktoken mocked from tests
* change progress ticker
* fix ruff check
* added cosmosdb constructor and database methods
* added rest of abstract method headers
* added cosmos db container methods
* implemented has and delete methods
* finished implementing abstract class methods
* integrated class into storage factory
* integrated cosmosdb class into cache factory
* added support for new config file fields
* replaced primary key cosmosdb initialization with connection strings
* modified cosmosdb setter to require json
* Fix non-default emitters
* Format
* Ruff
* ruff
* first successful run of cosmosdb indexing
* removed extraneous container_name setting
* require base_dir to be typed as str
* reverted merged changed from closed branch
* removed nested try statement
* readded initial non-parquet emitter fix
* added basic support for parquet emitter using internal conversions
* merged with main and resolved conflicts
* fixed more merge conflicts
* added cosmosdb functionality to query pipeline
* tested query for cosmosdb
* collapsed cosmosdb schema to use minimal containers and databases
* simplified create_database and create_container functions
* ruff fixes and semversioner
* spellcheck and ci fixes
* updated pyproject toml and lock file
* apply fixes after merge from main
* add temporary comments
* refactor cache factory
* refactored storage factory
* minor formatting
* update dictionary
* fix spellcheck typo
* fix default value
* fix pydantic model defaults
* update pydantic models
* fix init_content
* cleanup how factory passes parameters to file storage
* remove unnecessary output file type
* update pydantic model
* cleanup code
* implemented clear method
* fix merge from main
* add test stub for cosmosdb
* regenerate lock file
* modified set method to collapse parquet rows
* modified get method to collapse parquet rows
* updated has and delete methods and docstrings to adhere to new schema
* added prefix helper function
* replaced delimiter for prefixed id
* verified empty tests are passing
* fix merges from main
* add find test
* update cicd step name
* tested querying for new schema
* resolved errors from merge conflicts
* refactored set method to handle cache in new schema
* refactored get method to handle cache in new schema
* force unique ids to be written to cosmos for nodes
* found bug with has and delete methods
* modified has and delete to work with cache in new schema
* fix the merge from main
* minor typo fixes
* update lock file
* spellcheck fix
* fix init function signature
* minor formatting updates
* remove https protocol
* change localhost to 127.0.0.1 address
* update pytest to use bacj engine
* verified cache tests
* improved speed of has function
* resolved pytest error with find function
* added test for child method
* make container_name variable private as _container_name
* minor variable name fix
* cleanup cosmos pytest and make the cosmosdb storage class operations more efficient
* update cicd to use different cosmosdb emulator
* test with http protocol
* added pytest for clear()
* add longer timeout for cosmosdb emulator startup
* revert http connection back to https
* add comments to cicd code for future dev usage
* set to container and database clients to none upon deletion
* ruff changes
* add comments to cicd code
* removed unneeded None statements and ruff fixes
* more ruff fixes
* Update test_run.py
* remove unnecessary call to delete container
* ruff format updates
* Reverted test_run.py
* fix ruff formatter errors
* cleanup variable names to be more consistent
* remove extra semversioner file
* revert pydantic model changes
* revert pydantic model change
* revert pydantic model change
* re-enable inline formatting rule
* update documentation in dev guide
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
Co-authored-by: Josh Bradley <joshbradley@microsoft.com>
* Firsst cut at config cleanup
* Reorder top nav
* Add query prompts to tuning page
* Remove dynamic notebook from nav
* Add more thorough yml config descriptions in docs
* Further clean out the config
* Semver
* Add new blog post
* Emphasize yaml
* Clarify output
* Fix unit test
* Fix bullet nesting
* Add source documents for verb tests
* Remove entity_type erroneous column
* Add new test data
* Remove source/target degree columns
* Remove top_level_node_id
* Remove chunk column configs
* Rename "chunk" to "text"
* Rename "chunk" to "text" in base
* Re-map document input to use base text units
* Revert base text units as final documents dep
* Update test data
* Split/rename node source_id
* Drop node size (dup of degree)
* Drop document_ids from covariates
* Remove unused document_ids from models
* Remove n_tokens from covariate table
* Fix missed document_ids delete
* Wire base text units to final documents
* Rename relationship rank as combined_degree
* Add rank as first-class property to Relationship
* Remove split_text operation
* Fix relationships test parquet
* Update test parquets
* Add entity ids to community table
* Remove stored graph embedding columns
* Format
* Semver
* Fix JSON typo
* Spelling
* Rename lancedb
* Sort lancedb
* Fix unit test
* Fix test to account for changing period
* Update tests for separate embeddings
* Format
* Better assertion printing
* Fix unit test for windows
* Rename document.raw_content -> document.text
* Remove read_documents function
* Remove unused document summary from model
* Remove unused imports
* Format
* Add new snapshots to default init
* Use util to construct embeddings collection name
* Align inc index model with branch changes
* Update data and tests for int ids
* Clean up embedding locs
* Switch entity "name" to "title" for consistency
* Fix short_id -> human_readable_id defaults
* Format
* Rework community IDs
* Fix community size compute
* Fix unit tests
* Fix report read
* Pare down nodes table output
* Fix unit test
* Fix merge
* Fix community loading
* Format
* Fix community id report extraction
* Update tests
* Consistent short IDs and ordering
* Update ordering and tests
* Update incremental for new nodes model
* Guard document columns loc
* Match column ordering
* Fix document guard
* Update smoke tests
* Fill NA on community extract
* Logging for smoke test debug
* Add parquet schema details doc
* Fix community hierarchy guard
* Use better empty hierarchy guard
* Back-compat shims
* Semver
* Fix warning
* Format
* Remove default fallback
* Reuse key
* Remove excess vars from gh-pages build
* Delete redundant javascript ci
* Pull apart testing CI
* Clean up integration tests build
* Move storage tests to integration CI
* Take py 3.10 out of smoke tests matrix
* Use minimum supported python version for most tests
* Re-run main CI on any test change
* Add Josh and Kenny to author list
* Update auto-resolve perms
* Add support for both float and int on schema validation for community report generation
* Cast instead of type check
* Add mising file
* Add prompt with ints to smoke tests
* Fix unit tests
* Fix unit tests
* Fix sort_context max_tokens & max_tokens param in verb
* Fix sort_context for windows test
* add semversioner file
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
