NadezhdaNovotortseva
4868decc3b
fix: airflow ingest get_pipelines_list order_by added ( #22909 )
...
* fix: airflow ingest get_pipelines_list order_by added
* code reformatted
---------
Co-authored-by: Надежда Коцюба <nadezhda.kotsyuba@uni.rest>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
(cherry picked from commit 24e5155ffd01f73143dc05ce1c14956bc9ae70fd)
2025-08-24 19:50:34 +00:00
Oliver Schlüter
3adf6b38b9
Follow-up to #22474 : Expose retry_codes parameter in Airflow lineage backend configuration ( #22994 )
...
(cherry picked from commit b29deea3dedd0b3044486230377d4474ed04531d)
2025-08-24 19:45:50 +00:00
Mayur Singal
155241605e
MINOR: Improve Databricks Profiler & Test Connection ( #22732 )
2025-08-22 17:35:49 +05:30
Keshav Mohta
da67de51cc
Fix #18491 : ingestion fails for Iceberg tables with nested partition column ( #23031 )
...
* fix: ingestion fails for Iceberg tables with nested partition column
* test: added test to cover nested partition column for iceberg
* refactor: used if-else in tablePartition check
* fix: partition_column_name & column_partition_type typo
(cherry picked from commit 2f655daedc1bb82fbb97bec0e42fca15a8bb7863)
2025-08-22 12:01:21 +00:00
Keshav Mohta
ddc9e307be
Fixes: Snowflake Tags Ingestion ( #23040 )
...
* fix: snowflake tag ingestion error handling
* fix: python checkstyle
(cherry picked from commit 697b318f75ef630907abc2dece428ec67383ab53)
2025-08-22 11:59:55 +00:00
Ayush Shah
c99edbe290
Fixes #21677 : Refactor and enhance the entity name transformation logic ( #22695 )
2025-08-22 15:49:31 +05:30
github-actions
621302b8c7
chore(release): Prepare Branch for 1.9.3
2025-08-22 06:34:54 +00:00
Mohit Tilala
4f78c1d1b7
Fixes #22112 : Snowflake schema tags inheritance ( #22979 )
...
* Add schema-level tags and tag inheritance support for snowflake
* Add tests for schema tag inheritance
* Lint fixes
(cherry picked from commit 26fedbaf0eccb774837791df6b3337643c72014d)
2025-08-20 04:24:09 +00:00
Mohit Tilala
3a22bb7b38
Fixes #22238 : [SAP HANA] Correction of physical schema mapping and column lookup at each layer of calculation view ( #22952 )
...
(cherry picked from commit cc4b3574444fd41a2b46ac5f19666d1e96a18294)
2025-08-19 13:16:38 +00:00
Keshav Mohta
5bcb68e02d
Fix #22340 : Execution Time Support for NiFi Connector ( #22981 )
...
* feat: added nifi execution history
* doc: added pipeline status in available features
(cherry picked from commit f39f57ddcd44f0b2079f9363469f1d2e0112bc3b)
2025-08-19 08:40:13 +00:00
Teddy
c69c21d82e
ISSUE #1753 - Add Row Count to Custom SQL Test ( #22697 )
...
* feat: add count rows support for custom SQL
* style: ran python linting
* feat: added logic for partitioned custom sql row count
* migration: partitionExpression parameter
* chore: resolve conflicts
(cherry picked from commit d58b8a63d675e9bf91a2283a5f37702648cdab7f)
2025-08-19 10:25:15 +02:00
Ariel Schulz
8d98833622
Feature/1 fix and add lineage to exasol connector ( #21399 )
...
* Add lineage to Exasol connector
* Update test_connection to return TestConnectionResult
* Add exasol tests & dependencies to tests in setup.py
* Opensearch is required for testing, so add it there
* Modify metadata
* Update documentation for lineage
* Apply formatting changes to code
* Apply make py_format
2025-08-19 13:27:02 +05:30
github-actions
d17042a90e
chore(release): Prepare Branch for 1.9.2
2025-08-14 16:12:26 +00:00
Mayur Singal
288c655436
MINOR: Fix usage failure after cross lineage ( #22926 )
...
(cherry picked from commit e74de5df81277bf9f0af94f128c60821d5ab23ca)
2025-08-13 16:48:27 +05:30
harshsoni2024
3db10dc98b
precede source table name before pbi table name ( #22902 )
2025-08-13 10:00:27 +05:30
Mayur Singal
ddac981b54
Fix #22164 : Fix airflow compatibility issue with version 2.2.5 ( #22897 )
...
(cherry picked from commit 23959bd21a2a7428b738163dbe35a4198ea34561)
2025-08-12 13:03:45 +00:00
Mayur Singal
f4127757c3
FIX #21180 : Implement Cross Service Lineage ( #22665 )
2025-08-12 18:23:36 +05:30
Copilot
a4da357892
Add OpenAPI YAML format support for REST API ingestion ( #22304 )
...
* Initial plan
* Implement OpenAPI YAML support with backward JSON compatibility
Co-authored-by: harshach <38649+harshach@users.noreply.github.com>
* fix tests & lint
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: harshach <38649+harshach@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2025-08-12 18:23:14 +05:30
harshsoni2024
ae1a156d76
issue-22425: PowerBI parse expression along with measure ( #22870 )
2025-08-12 17:07:21 +05:30
Sriharsha Chintalapani
08d03b548b
Fix #1093 : Add Grafana Support ( #22571 )
...
* Fix #1093 : Add Grafana Support
* Update generated TypeScript types
* Grafana test fix
* Update
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Mac.lan>
2025-08-12 15:57:54 +05:30
github-actions
e16dd73084
chore(release): Prepare Branch for 1.9.1
2025-08-12 05:09:26 +00:00
Sriharsha Chintalapani
f01c44fe23
fix logger level in sample_data.yaml ( #22867 )
...
(cherry picked from commit c4d395d14daf1fa539a79dd8b6f5fe732b19de82)
2025-08-11 06:56:39 +00:00
Ayush Shah
4dc2e77bbc
Add DriveService support in ingestion routes
...
* Introduced CreateDriveServiceRequest to handle drive service creation
* Updated routes.py to include DriveService entity
This enhancement expands the ingestion framework's capabilities to support drive services, aligning with the project's goal of comprehensive metadata management.
2025-08-07 10:39:39 +05:30
Akash Verma
a403879f86
feature: googlesheet connector ( #22464 )
...
* feature: googlesheet connector
* updates
* minor change
* java checkstyle
* googledrive files
* fix
* make the google sheets work
* fix directory and files
* remove ui
* update generated types
---------
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Mac.lan>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2025-08-06 22:53:42 +05:30
github-actions
781d1f4883
chore(release): Prepare Branch for 1.9.0
2025-08-04 07:45:41 +00:00
Mayur Singal
b74e181d52
MINOR: Improve Unity Catalog Usage ( #22721 )
2025-08-04 11:04:10 +05:30
Mayur Singal
fe28faa13f
MINOR: Add support for csv.gz in datalake ( #22666 )
...
* MINOR: Add support for csv.gz in datalake
* fileformat change
* Update generated TypeScript types
* pyformat
2025-08-01 17:39:19 +05:30
Mayur Singal
dfb7eeaaaa
MINOR: Improve datalake processing table log ( #22631 )
2025-08-01 15:07:37 +05:30
Nick Acosta
0a395ea487
Fixed typo referring to airbyte in Fivetran file ( #22686 )
...
Co-authored-by: Nick Acosta <nick.acosta@getcollate.io>
2025-08-01 14:49:23 +05:30
IceS2
7f8298d49e
Update DataLake and PostgreSQL connection ( #22682 )
2025-08-01 11:08:43 +02:00
Pere Miquel Brull
9cb146591c
MINOR - Add Data Contract Alerts ( #22671 )
...
* WIP - MINOR - Rule Engine
* WIP - MINOR - Rule Engine
* WIP - MINOR - Rule Engine
* WIP - MINOR - Rule Engine
* rules
* rules
* rules
* fix retrieval by entity
* test dc
* test dc
* WIP: Data contract feature
* destructure component to it's own files
* WIP contract tab
* update local
* fix test
* First iteration for multi domain support
* fix inheritance fields
* fix inheritance fields
* fix create interface
* fix few more tests
* fix indexing updates
* fix domain rel
* update domain --> domains
* merge
* fix merge
* fix csv tests and createEntity interface
* Update generated TypeScript types
* Trigger Build
* migrations
* fix tests
* fix tests
* fix tests
* Update generated TypeScript types
* Trigger Build
* handle drive service
* fix pg migration
* fix domains ref after merge and clean python tests
* Update generated TypeScript types
* fix merge domains
* format
* add missing migrations
* Update generated TypeScript types
* tests
* Update generated TypeScript types
* Trigger Build
* tests
* tests
* fix py test
* migrate domain to domains and fix compilation errors
* fix domain assignement
* fix domain spec
* fix py tests
* fix data product creation issue
* fix domain tests
* fix bulk import
* fix tests
* fix tests
* fix query and domain migration
* fix py test
* fix playwrights
* fix getEntitiesWithDisplayName indexing quotes
* fix domain prapogation tests
* execute dc
* execute dc
* fix domain propagation
* Fix patch api
* fix domain schema build edit playwright
* fix test
* fix test
* fix domain selection issue and console errors
* handle semantics validation
* handle ingestion pipelines for DQ
* prepare for getting DQ status
* get last result
* update schema and prep sample data
* test
* handle data contracts tests in SDK
* handle data contracts tests in SDK
* execute test suite
* execute test suite
* add schema validations
* add schema validations
* add schema validations
* add schema validations
* handle data contracts tests in SDK
* add schema validations
* comments
* generate schemas
* add fields
* MINOR - Data Contract Alerts
* MINOR - Data Contract Alerts
* MINOR - Data Contract Alerts
* Fix url
* tests
* minor ui fixes around localization keys
* supporting locaization keys
* fix test with updated response
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-08-01 06:44:47 +02:00
MarkSaf17
58078f582c
[WIP] Issue 22627 ( #22647 )
...
* [ISSUE-22627] fix(dbt): Support documented owner configuration with backward compatibility
* typo
* added tests
---------
Co-authored-by: marsafronov <marsafronov@ecom.tech>
Co-authored-by: SumanMaharana <sumanmaharana786@gmail.com>
2025-07-31 11:02:18 +00:00
IceS2
f578a81277
MINOR: Update table profile config to add spark configs. Update spark config to add temp path ( #22646 )
...
* Update table profile config to add spark configs. Update spark config to add temp path
* Add default null value for sparkTableProfilerConfig
* Fix TableProfileConfig extension DAO query
* Update generated TypeScript types
* Implemented Dependency Injection for the ProfilerProcessorConfig
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-07-31 11:50:30 +02:00
Roman Sheludko
e098635842
fix: Remove redundant set of braces in additional_config ( #22640 )
...
After the changes in https://github.com/open-metadata/OpenMetadata/pull/22474 we've started to experience the following error:
2025-07-31 10:22:45 +02:00
Keshav Mohta
f8b588e6f0
Feature: Security Service ( #22450 )
...
* feat: security service initial code
* support secrity type for service from ui
* refactor: added missing java files for security service and require json files
* refactor: added doc file, oneOf java changes for ranger
* Merge branch 'main' of https://github.com/open-metadata/OpenMetadata into issue-20145
* update the security services schema to have domains instead of domain
* fix: added security service metadata pipeline
* Make changes for security service UI
* Update missing translation
* update type
* Fix the java checkstyle
* fix: added ingestionRunner in security service json
* Hide security service for OSS
* fix: reverse metadata documentation
* refactor: added securityConnection in test connection
* fix: documentation and ts file changes
* fix: deleted __init__ file and added SSISConnection again in the workflowclassconverter.java
* fix: convert domain to domains
* fix: added new file for basicAuth for ranger and change in rangerConnectionClassConverter.java file
* fix: java checkstyle
* fix: mysql security service sql
* fix: security service index mapping
* fix: moved migrations in new version and fix classConverterFactory
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-07-31 06:38:21 +02:00
Pere Miquel Brull
dfe3fd6357
MINOR - Data Contract Validation ( #22541 )
2025-07-30 23:01:27 +02:00
Ariel Schulz
798a20afd8
Fix test_connection to adhere to OMD standards for DB connections ( #22478 )
2025-07-31 00:29:03 +05:30
Suman Maharana
3a90b38a26
Fix: Tableau ca cert auth ( #22041 )
...
* Fix: Tableau ca cert auth
* py_format
* Added ssl tests
* fix lint errors
2025-07-30 09:38:47 +05:30
harshsoni2024
50428e2e7b
feat-22574: Datalake ingestion fix for larger files ( #22575 )
2025-07-29 20:40:19 +05:30
Suman Maharana
670dc53b46
Minor: fix tableau handle none entities ( #22630 )
...
* Minor: fix tableau handle none entities
* added tests
2025-07-29 13:58:11 +02:00
Ayush Shah
987e3ed9e0
Fixes #22144 : Enhance memory management and proper cleanup of resources in profiler ( #22438 )
2025-07-29 16:22:21 +05:30
Mayur Singal
199e3b981c
Fix #14830 : Ignore non current columns for iceberg tables for glue & athena ( #22564 )
2025-07-29 16:19:09 +05:30
Mayur Singal
cc9506db20
MINOR: Postgres Implement schema fallback ( #21858 )
...
* MINOR: Postgres Implement schema fallback
* missing sql_lineage file
2025-07-29 14:45:21 +05:30
Suman Maharana
54dcdc7d82
Fix #20689 : Trino Column validation errors for highly complex fields ( #22421 )
...
* Fix: Trino Column validation errors for highly complex fields
* addressed copilot comms
* fixed tests
* fixed tests and addressed comms
* missed file
2025-07-28 11:11:44 +05:30
IceS2
bad772db39
FIX #22099 : enable 'Column values to be in set' test case for boolean columns ( #22491 )
...
* fix(dq): enable ''Column values to be in set'' test case for boolean columns
Add BOOLEAN to supportedDataTypes array in columnValuesToBeInSet.json
to allow boolean column validation with predefined allowed values.
This enables users to enforce strict true/false validation on boolean
columns directly at the column level, resolving issue #22099 .
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Add tests to the new feature
* Add migrations and columnValuesToBeNotInSet
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
2025-07-25 15:17:38 +02:00
Ayush Shah
1e8e38f2ca
MINOR: Custom properties Data types fix ( #22342 )
2025-07-25 18:39:53 +05:30
Mayur Singal
37b10a102f
MINOR: Improve ometa logging ( #22586 )
2025-07-25 18:26:44 +05:30
Mayur Singal
b8db86bc4f
MINOR: Fix airflow ingestion for older version ( #22581 )
2025-07-25 18:22:33 +05:30
Sriharsha Chintalapani
b0586f849f
Fix #22511 : k8s secret support for Secrets Manager ( #22516 )
...
* Fix #22511 : k8s secret support for Secrets Manager
* Update generated TypeScript types
* address comments
* pylint fix
* fix java checkstyle
* improve inCluster description in schema
* fix failing tests
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2025-07-24 12:40:51 +02:00
Mayur Singal
a94c1bef47
MINOR: Fix mysql pytest version ( #22535 )
2025-07-24 09:45:37 +05:30
