* fix(airflow): correctly extract owners from serialized Airflow DAGs
Airflow serialization format wraps tasks under `__var` and `__type`.
Previously, the OpenMetadata Airflow connector failed to extract task owners properly in this format.
This patch:
- Flattens `__var` when parsing task owners
- Fallbacks to `default_args["owner"]` if no task-level owner is explicitly present
- Ensures correct DAG owner is picked as the most common task owner
- Handles compatibility with older Airflow versions
Fixes: #21106
* test(airflow): add tests for owner extraction from serialized Airflow DAGs
Adds new test cases to validate owner extraction logic:
- Owners from serialized task format (`__var`)
- Fallback to `default_args['owner']` if task owners are missing
- Resolution of most common owner
- Compatibility with unstructured or missing owners
* remove test version specific comment
* simplify comments and warnings
* fix return statement
* fixing formatting
* adding handling of default args
* fixing and adding more tests
* feat: add support for DBX system metrics
* feat: add support for DBX system metrics
* fix: added WRITE back
* fix: failing test cases
* fix: failing test
* Fix : query_cost_record_search_index Search exception for elasticsearch instance
* add sample query to cover test scenarios
* update mapping and fix test
* Update Profile Workflow to allow engine configuration
* Add ui generated schemas
* Add Repository Override mechanism based on annotations
* Implement logic to use the ProcessingEngine configuration
* Update SparkEngine to use remote and not master
* metadata dbt
* fix:
- default path to current directory
- addional warning and exception handling for missing metadata config vars
* test: add unit tests for DBT Ingestion CLI
* refactor
* PR review:
- using Pydantic to parse and validate the openmetadata config in dbt's .yml
- extended test-cases
- giving user more configuration options for ingestion
* py refactoring
* add: dbt-auto ingest docs
* Improvements:
- using environement variables for loading sensitve variables
- added docs for auto dbt-ingestion for dbt-core
- more test cases
* fix:
- test case for reading JWT token inside the the method
* refactor: py code formatting
* refactor: py formatting
* ingest-dbt docs updated
* refined test cases
* Chore:
- sonar vulnerability issue review
- using existing URL class for host validation
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster
* Add exact match configs
* Add exact match configs
* Update Logic to build search source builder with exact match priority
* Revert "Update Logic to build search source builder with exact match priority"
This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.
* Revert "Add exact match configs"
This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.
* revert display name change
* make boost mode as sum by defaul
* add more fqnparts for schema and database
* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue
* use fqn split
* refactor fqn parsing
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster
* Add exact match configs
* Add exact match configs
* Update Logic to build search source builder with exact match priority
* Revert "Update Logic to build search source builder with exact match priority"
This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.
* Revert "Add exact match configs"
This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.
* revert display name change
* make boost mode as sum by defaul
* add more fqnparts for schema and database
* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue
* use fqn split
* Refactor FQN Parts
---------
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* Initial implementation for our Connection Class
* Implement the Initial Connection class
* Add Unit Tests
* Implement Dependency Injection for the Ingestion Framework
* Fix Test
* Fix Profile Test Connection
* Add Injection to Metrics in Profiler
* Add Injection to the Profiler
* Fix UnitTests
* Fix Pytests
* Fix Tests
* Fix types
* Initial implementation for our Connection Class
* Implement the Initial Connection class
* Add Unit Tests
* Implement Dependency Injection for the Ingestion Framework
* Fix Test
* Fix Profile Test Connection
* Fix test, making the injection test run last
* Update connections.py
* Changed NewType to an AbstractClass to avoid linting issues
* remove comment
* Fix bug in service spec
* Update PyTest version to avoid importlib.reader wrong import
* Initial implementation for our Connection Class
* Implement the Initial Connection class
* Add Unit Tests
* Fix Test
* Fix Profile Test Connection
* Remove unit test
* Remove comment
* Fix tests and missing changes
