Mayur Singal
88115e1218
MINOR: Fix training / issue in UC S3 lineage ( #23816 )
2025-10-09 18:44:07 +02:00
Mayur Singal
05f064787f
Feat: Add kafka lineage support in databricks pipelines ( #23813 )
...
* Add dlt pipeline support
* Fix code style
* Add variable parsing
* Fix kafka lineage
---------
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2025-10-09 16:42:08 +02:00
Sriharsha Chintalapani
454d7367b0
Kafka Connect: Support Confluent Cloud connectors ( #23780 )
2025-10-09 01:28:27 +05:30
Mayur Singal
4708c2b64f
feat: Unity Catalog Lineage Enhancement: External Location Support ( #23790 )
2025-10-08 20:26:39 +05:30
harshsoni2024
f2819ce4e4
Fix: PowerBI snowflake query lineage parsing ( #23746 )
2025-10-08 18:32:25 +05:30
harshsoni2024
da7a2778f6
MINOR: iceberg load table retry backoff ( #23579 )
2025-10-05 23:42:56 +05:30
Sriharsha Chintalapani
fc7412f6dd
Add Timescale Connector ( #23665 )
...
* Add Timescale Connector
* Update generated TypeScript types
* Add UI changes for the Timescale
* lineage, usage and java
* Add beta tag
* update logo
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Akash Verma <akashverma@Mac.lan>
2025-10-03 19:00:59 -07:00
Keshav Mohta
3d49b6689d
Fixes #23356 : Databricks & UnityCatalog OAuth and Azure AD Auth ( #23561 )
...
* feat: databricks oauth and azure ad auth setup
* refactor: add auth type changes in databricks.md
* fix: test after oauth changes
* refactor: unity catalog connection to databricks connection code
* feat: added oauth and azure ad for unity catalog
* fix: unitycatalog tests, doc & required type in connection.json
* fix: generated tx files
* fix: exporter databricksConnection file
* refactor: unitycatalog example file
* fix: usage example files
* fix: unity catalog sqlalchemy connection
* fix: unity catalog client headers
* refactor: make common auth.py for dbx and unitycatalog
* fix: auth functions import
* fix: test unity catalog tags as None
* fix: type hinting and sql migration
* fix: migration for postgres
2025-10-03 19:53:19 +05:30
Akash Verma
06453a925d
Fix #21093 : Update test connection improvements ( #23516 )
...
* Update test connection improvements
* Update queries
* checkstyle
* fix test failure
---------
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
2025-10-03 13:50:46 +05:30
Suman Maharana
c8055576ba
Fixes #21686 : Add missing includeOwners check in dashboard services ( #22514 )
2025-10-03 10:53:25 +05:30
Mohit Tilala
22a0925cd2
Fix correct snowflake object types in source url ( #23612 )
2025-09-29 15:31:10 +00:00
Keshav Mohta
cb26c91442
Revert "Fixes #23356 : Databricks OAuth & Azure AD Auth ( #23482 )" ( #23530 )
...
This reverts commit f1afe8f5f114ee58090168fd7ae5d66b38a01ab0.
2025-09-23 17:44:16 +02:00
Keshav Mohta
f1afe8f5f1
Fixes #23356 : Databricks OAuth & Azure AD Auth ( #23482 )
...
* feat: databricks oauth and azure ad auth setup
* refactor: add auth type changes in databricks.md
* fix: test after oauth changes
* refactor: unity catalog connection to databricks connection code
2025-09-23 15:22:50 +05:30
Keshav Mohta
9262040381
fix: handle database native types for create table request during openlineage lineage ( #23513 )
2025-09-23 10:11:39 +02:00
Suman Maharana
e2b903532e
Fixes - Kafkaconnect lineage & descriptions ( #23234 )
...
* Fix Kafkaconnect lineage & descriptions
* fix typos
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* address comments
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* address comms
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-23 10:08:37 +02:00
Mohit Tilala
d1e60acd2a
[SAP HANA] Prevent exponential processing lineage parsing and use full name for filtering ( #23484 )
...
* Prevent exponential processing lineage parsing
* Use full name of views for filtering
* pylint fix - isort
2025-09-22 19:46:34 +05:30
Keshav Mohta
1a67e4fb7d
Feature: MariaDB Stored Procedures and Functions Support #23422
2025-09-18 17:59:39 +05:30
Akash Verma
da5dab7fef
Fixes #23388 : Handle string and dict types for Metabase dataset_query field ( #23417 )
...
* Handle string and dict types for Metabase dataset_query field
* Added tests
---------
Co-authored-by: Akash Verma <akashverma@Mac.lan>
2025-09-16 16:57:08 -07:00
Mohit Tilala
f2fd8a9107
Fixes #22452 : [Snowflake] Add custom host support for View in Snowflake source url ( #23209 )
2025-09-03 14:13:03 +05:30
Suman Maharana
20e18d4f9f
Add ssl support to hive ( #22831 )
...
* Add ssl support to hive
* Added missing ts files
* Added version to pure transport
* Added Tests
* fix tests add missing files
2025-09-02 20:13:30 +05:30
Mayur Singal
08ee62a198
MINOR: Add Unstructured Formats Support to GCS Storage Connector ( #23158 )
2025-09-02 18:22:39 +05:30
Mohit Tilala
744494968e
Fixes #22238 : [SAP HANA] Add calculated view columns' formula parsing logic ( #23017 )
...
* Add calculated view columns' formula parsing logic with correct source reference
* Handle top level column formula parsing and pass formula expression in column lineage detail
---------
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
2025-08-26 07:19:11 +05:30
Keshav Mohta
2f655daedc
Fix #18491 : ingestion fails for Iceberg tables with nested partition column ( #23031 )
...
* fix: ingestion fails for Iceberg tables with nested partition column
* test: added test to cover nested partition column for iceberg
* refactor: used if-else in tablePartition check
* fix: partition_column_name & column_partition_type typo
2025-08-22 17:25:59 +05:30
Mohit Tilala
26fedbaf0e
Fixes #22112 : Snowflake schema tags inheritance ( #22979 )
...
* Add schema-level tags and tag inheritance support for snowflake
* Add tests for schema tag inheritance
* Lint fixes
2025-08-20 09:52:44 +05:30
Mohit Tilala
cc4b357444
Fixes #22238 : [SAP HANA] Correction of physical schema mapping and column lookup at each layer of calculation view ( #22952 )
2025-08-19 18:45:06 +05:30
Copilot
8cc9d2af71
Add OpenAPI YAML format support for REST API ingestion ( #22304 )
...
* Initial plan
* Implement OpenAPI YAML support with backward JSON compatibility
Co-authored-by: harshach <38649+harshach@users.noreply.github.com>
* fix tests & lint
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: harshach <38649+harshach@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2025-08-12 18:22:41 +05:30
Sriharsha Chintalapani
15b92735b9
Fix #1093 : Add Grafana Support ( #22571 )
...
* Fix #1093 : Add Grafana Support
* Update generated TypeScript types
* Grafana test fix
* Update
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Mac.lan>
2025-08-11 19:39:39 +05:30
Ariel Schulz
d31e2d8ba0
Feature/1 fix and add lineage to exasol connector ( #21399 )
...
* Add lineage to Exasol connector
* Update test_connection to return TestConnectionResult
* Add exasol tests & dependencies to tests in setup.py
* Opensearch is required for testing, so add it there
* Modify metadata
* Update documentation for lineage
* Apply formatting changes to code
* Apply make py_format
2025-08-06 23:49:38 +05:30
Mayur Singal
00b6da5b84
MINOR: Improve Databricks Profiler & Test Connection ( #22732 )
2025-08-06 00:41:11 +05:30
Suman Maharana
3a90b38a26
Fix: Tableau ca cert auth ( #22041 )
...
* Fix: Tableau ca cert auth
* py_format
* Added ssl tests
* fix lint errors
2025-07-30 09:38:47 +05:30
Suman Maharana
670dc53b46
Minor: fix tableau handle none entities ( #22630 )
...
* Minor: fix tableau handle none entities
* added tests
2025-07-29 13:58:11 +02:00
Mayur Singal
199e3b981c
Fix #14830 : Ignore non current columns for iceberg tables for glue & athena ( #22564 )
2025-07-29 16:19:09 +05:30
Mayur Singal
b8db86bc4f
MINOR: Fix airflow ingestion for older version ( #22581 )
2025-07-25 18:22:33 +05:30
Chirag Madlani
b098395602
Data contracts support for tables & Multi Domain Migration ( #22108 )
...
* WIP - MINOR - Rule Engine
* WIP - MINOR - Rule Engine
* WIP - MINOR - Rule Engine
* WIP - MINOR - Rule Engine
* rules
* rules
* rules
* fix retrieval by entity
* test dc
* test dc
* WIP: Data contract feature
* destructure component to it's own files
* WIP contract tab
* update local
* fix test
* First iteration for multi domain support
* fix inheritance fields
* fix inheritance fields
* fix create interface
* fix few more tests
* fix indexing updates
* fix domain rel
* update domain --> domains
* merge
* fix merge
* fix csv tests and createEntity interface
* Update generated TypeScript types
* Trigger Build
* migrations
* fix tests
* fix tests
* fix tests
* Update generated TypeScript types
* Trigger Build
* handle drive service
* fix pg migration
* fix domains ref after merge and clean python tests
* Update generated TypeScript types
* fix merge domains
* format
* add missing migrations
* Update generated TypeScript types
* tests
* Update generated TypeScript types
* Trigger Build
* tests
* tests
* fix py test
* migrate domain to domains and fix compilation errors
* fix domain assignement
* fix domain spec
* fix py tests
* fix data product creation issue
* fix domain tests
* fix bulk import
* fix tests
* fix tests
* fix query and domain migration
* fix py test
* fix playwrights
* fix getEntitiesWithDisplayName indexing quotes
* fix domain prapogation tests
* fix domain propagation
* Fix patch api
* fix domain schema build edit playwright
* fix test
* fix test
* fix domain selection issue and console errors
* quick fix landing page changes
* fix remaining tests
* fix ui tests
* Fix adding data products
* format
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-07-22 09:34:50 +02:00
Suman Maharana
9838278ac4
Add: Schema and Database Mark Deletion ( #22088 )
...
* Added Schema and Database Mark Deletion
* removed unnecessary changes
* fixed marked deleted databases
* Added to all db connectors
* Added generated types
* Added tests
2025-07-15 16:26:46 +02:00
Mayur Singal
47b20a5f2d
MINOR: Fix databricks default schema issue ( #22254 )
2025-07-09 11:50:50 -07:00
Keshav Mohta
6e40f976e7
Fix #20145 : Implemented Prefix For Dashboard Service ( #21585 )
...
* feat: implemented microstrategy lineage & dbServicePrefix
* feat: added dbServicePrefixes support in other dashboards
* fix: test_metabase and powerbi extra code remove
* fix: python checkstyle
* refactor: added prefix support for other connectors - superset, tableau, etc
* refactor: added migration for prefix change and fix dbServicePrefixes field description
* refactor: added prefix changes in superset db source
* doc: add prefix in tableau doc
* fix: typescript files and postgres migration for prefix
* fix: moved migration in 1.8.2
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-07-08 18:54:35 +02:00
Mayur Singal
2fcf3281d8
MINOR: Fix snowflake map key type error ( #22205 )
2025-07-08 14:44:05 +05:30
Ferjani Nasraoui
b0e1a136cf
Fixes #21106 : Support owner extraction from serialized Airflow DAGs ( #22071 )
...
* fix(airflow): correctly extract owners from serialized Airflow DAGs
Airflow serialization format wraps tasks under `__var` and `__type`.
Previously, the OpenMetadata Airflow connector failed to extract task owners properly in this format.
This patch:
- Flattens `__var` when parsing task owners
- Fallbacks to `default_args["owner"]` if no task-level owner is explicitly present
- Ensures correct DAG owner is picked as the most common task owner
- Handles compatibility with older Airflow versions
Fixes : #21106
* test(airflow): add tests for owner extraction from serialized Airflow DAGs
Adds new test cases to validate owner extraction logic:
- Owners from serialized task format (`__var`)
- Fallback to `default_args['owner']` if task owners are missing
- Resolution of most common owner
- Compatibility with unstructured or missing owners
* remove test version specific comment
* simplify comments and warnings
* fix return statement
* fixing formatting
* adding handling of default args
* fixing and adding more tests
2025-07-03 14:21:36 +05:30
Suman Maharana
e36e5da26e
Added Databricks pipeline Lineage ( #22014 )
2025-06-30 10:41:22 +05:30
harshsoni2024
10b377590c
qlikcloud get script tables ( #22022 )
2025-06-30 10:36:57 +05:30
harshsoni2024
616579a6c1
feat-21984: REST service process nested objects inside array dtype in schema ( #21984 )
2025-06-27 10:44:35 +05:30
harshsoni2024
f490406968
MINOR: pbi improve logging ( #21868 )
2025-06-20 16:32:56 +05:30
Keshav Mohta
73ea60b898
Refactor: Unity Catalog ( #21801 )
2025-06-20 16:04:34 +05:30
Suman Maharana
7be62f3ed9
Add: Tableau Hierarchy project filter ( #21811 )
2025-06-19 11:18:52 +05:30
harshsoni2024
d38ee0ed52
feat-21712: PowerBI internal entities & cross workspace lineage ( #21837 )
2025-06-18 20:46:17 +05:30
Keshav Mohta
7c0eeef049
Fixes #19692 : Implemented Nifi Pipeline Lineage ( #21802 )
...
* feat: implemented nifi pipeline lineage
* test: implemented tests for nifi pipeline lineage
* fix: yield_pipeline_bulk_lineage_details output type hinting
* fix: component check in connections
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2025-06-18 07:31:04 +00:00
harshsoni2024
a09a696358
MINOR: Tableau proxy url for sourceurl ( #21799 )
2025-06-18 10:52:08 +05:30
harshsoni2024
6a6180b2e3
powerbi change owner condition ( #21724 )
2025-06-12 16:11:43 +05:30
Suman Maharana
18f9f2cdb6
Fix: Tableau project id should always be a string ( #21700 )
2025-06-12 11:21:53 +05:30
