* update index api to accept callbacks
* fix hardcoded folder name that was creating an empty folder
* add API notebook
* add semversioner file
* filename change
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* switched hashing function helper to sha256
* refactored references to hashing util
* semversioner
* switched from sha256 to sha512
* new semversioner
* updated tests/verbs/data folder
* generated fresh parquet files in data folder
* moved ignore flag
* Fix local question gen and example notebook
* Update global search notebook
* Add lazy blog post
* Update breaking changes doc for migration notes
* Simplify Getting Started page
* Semver
* Spellcheck
* Fix types
* Add comments on cache-free migration
* Update wording
* Spelling
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Firsst cut at config cleanup
* Reorder top nav
* Add query prompts to tuning page
* Remove dynamic notebook from nav
* Add more thorough yml config descriptions in docs
* Further clean out the config
* Semver
* Add new blog post
* Emphasize yaml
* Clarify output
* Fix unit test
* Fix bullet nesting
* Fix footer contrast
* Fix broken links
* Remove a few unneeded examples
* Point python API example to the whole folder
* Convert schema bullets to tables
Updated the wording of the example scenario from "global search" to "drift search" to accurately reflect the topic. This improves clarity and ensures the documentation accurately describes its content.
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Move indexing prompts to root
* Move query prompts to root
* Export query prompts during init
* Extract general knowledge prompt
* Load query prompts from disk
* Semver
* Fix unit tests
* Add source documents for verb tests
* Remove entity_type erroneous column
* Add new test data
* Remove source/target degree columns
* Remove top_level_node_id
* Remove chunk column configs
* Rename "chunk" to "text"
* Rename "chunk" to "text" in base
* Re-map document input to use base text units
* Revert base text units as final documents dep
* Update test data
* Split/rename node source_id
* Drop node size (dup of degree)
* Drop document_ids from covariates
* Remove unused document_ids from models
* Remove n_tokens from covariate table
* Fix missed document_ids delete
* Wire base text units to final documents
* Rename relationship rank as combined_degree
* Add rank as first-class property to Relationship
* Remove split_text operation
* Fix relationships test parquet
* Update test parquets
* Add entity ids to community table
* Remove stored graph embedding columns
* Format
* Semver
* Fix JSON typo
* Spelling
* Rename lancedb
* Sort lancedb
* Fix unit test
* Fix test to account for changing period
* Update tests for separate embeddings
* Format
* Better assertion printing
* Fix unit test for windows
* Rename document.raw_content -> document.text
* Remove read_documents function
* Remove unused document summary from model
* Remove unused imports
* Format
* Add new snapshots to default init
* Use util to construct embeddings collection name
* Align inc index model with branch changes
* Update data and tests for int ids
* Clean up embedding locs
* Switch entity "name" to "title" for consistency
* Fix short_id -> human_readable_id defaults
* Format
* Rework community IDs
* Fix community size compute
* Fix unit tests
* Fix report read
* Pare down nodes table output
* Fix unit test
* Fix merge
* Fix community loading
* Format
* Fix community id report extraction
* Update tests
* Consistent short IDs and ordering
* Update ordering and tests
* Update incremental for new nodes model
* Guard document columns loc
* Match column ordering
* Fix document guard
* Update smoke tests
* Fill NA on community extract
* Logging for smoke test debug
* Add parquet schema details doc
* Fix community hierarchy guard
* Use better empty hierarchy guard
* Back-compat shims
* Semver
* Fix warning
* Format
* Remove default fallback
* Reuse key
* update gitignore
* add dynamic community sleection to updated main branch
* update SearchResult to record output_tokens.
* update search result
* dynamic search working
* format
* add llm_calls_categories and prompt_tokens and output_tokens cate
* update
* formatting
* log drift search output and prompt tokens separately
* update global_search.ipynb. update operate dulce dataset and add create_final_communities. update dynamic community selection init
* add .ipynb back to cspell.config.yaml
* format
* add notebook example on dynamic search
* rearrange
* update gitignore
* format code
* code format
* code format
* fix default variable
---------
Co-authored-by: Bryan Li <bryanlimy@gmail.com>
* Updated the variable names within the for-loop to differentiate between them and the original title variable used in the dataframe. This avoids corrupting the original column-name defined in the title variable.
* Semver and formart
---------
Co-authored-by: Gabriel Nieves-Ponce <gnievesponce@microsoft.com>
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
