yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-12-24 08:28:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: add note on column transformation logic (#13615)

Browse Source
This commit is contained in:
Harshal Sheth 2025-05-23 10:07:45 -07:00 committed by GitHub
parent 947761f875
commit e1de763c17
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 13 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
docs/how/updating-datahub.md
View File

@ -22,6 +22,18 @@ This file documents any backwards-incompatible changes in DataHub and assists pe
### Breaking Changes
### Known Issues
### Potential Downtime
### Deprecations
### Other Notable Changes
## 1.1.0
### Breaking Changes
- #13004: The `acryl-datahub-airflow-plugin` dropped support for Airflow 2.3 and 2.4.
- #13186: NoCode Migration Removed - This code hasn't been required in many years. If needed, a user should upgrade to DataHub 1.0.x prior to upgrading to a later version.
- #13397: `async_flag` removed from rest emitter, replaced with emit mode ASYNC
@ -45,6 +57,7 @@ This file documents any backwards-incompatible changes in DataHub and assists pe
- SYNC_PRIMARY: Synchronously updates the primary storage (SQL) but asynchronously updates search storage (Elasticsearch). Provides a balance between consistency and performance. Suitable for updates that need to be immediately reflected in direct entity retrievals but where search index consistency can be slightly delayed.
- ASYNC: Queues the metadata change for asynchronous processing and returns immediately. The client continues execution without waiting for the change to be fully processed. Best for high-throughput scenarios where eventual consistency is acceptable.
- ASYNC_WAIT: Queues the metadata change asynchronously but blocks until confirmation that the write has been fully persisted. More efficient than fully synchronous operations due to backend parallelization and batching while still providing strong consistency guarantees. Useful when you need confirmation of successful persistence without sacrificing performance.
- #13426 - Added support for extracting column transformation logic when using `use_queries_v2` with warehouse ingestion.
- #13499 - Added ELASTICSEARCH_MIN_SEARCH_FILTER_LENGTH configuration for ElasticSearch index config. If modified from the default, this configuration can have significant impact on search performance if changed and will trigger reindexing causing large delays in updates. Most users will not want to modify this.
## 1.0.0