1121 Commits

Author SHA1 Message Date
Sriharsha Chintalapani
fc7412f6dd
Add Timescale Connector (#23665)
* Add Timescale Connector

* Update generated TypeScript types

* Add UI changes for the Timescale

* lineage, usage and java

* Add beta tag

* update logo

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Akash Verma <akashverma@Mac.lan>
2025-10-03 19:00:59 -07:00
Keshav Mohta
6f47baa264
Fixes: Drive Service (#23637)
* fix: added missing fields

* fix: java checkstyle and add mimeType

* Fix Data Asset Headers for drive assets

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-10-03 17:06:30 +00:00
Keshav Mohta
3d49b6689d
Fixes #23356: Databricks & UnityCatalog OAuth and Azure AD Auth (#23561)
* feat: databricks oauth and azure ad auth setup

* refactor: add auth type changes in databricks.md

* fix: test after oauth changes

* refactor: unity catalog connection to databricks connection code

* feat: added oauth and azure ad for unity catalog

* fix: unitycatalog tests, doc & required type in connection.json

* fix: generated tx files

* fix: exporter databricksConnection file

* refactor: unitycatalog example file

* fix: usage example files

* fix: unity catalog sqlalchemy connection

* fix: unity catalog client headers

* refactor: make common auth.py for dbx and unitycatalog

* fix: auth functions import

* fix: test unity catalog tags as None

* fix: type hinting and sql migration

* fix: migration for postgres
2025-10-03 19:53:19 +05:30
Aniket Katkar
b7545e6305
Chore(UI): Improve advanced search field value suggestions (#23139)
* Update the index mappings

* Add the Data Product field for filtering
Improve the field suggestions shown for filter values

* Fix unit tests

* update the application locale file

* Update suggestion APIs for tags, certification and tier filters

* Add tests for the new suggestion changes.

* Add the version and dataProduct fields for index mapping back

* fix: update autocomplete queries to use structured query format for classification filtering

* Fix the classification not showing in search api response

* Fix the failing playwright Advanced search

* Add lowercase_normalizer for the tier field similar to the tag field in the index mappings

* Fix the Curated Assets test

* Fix the failing playwright tests

* Fix the playwright tests

* update the API await logic

* Fix the Advanced search tests

* Fix the translation file eslint

* Work on the comments

* Add data product display name normalization for jp and ru langs
2025-10-03 17:44:13 +05:30
Aniket Katkar
453c57df9f
Fix: Superset add service form bug (#23569)
* fix: remove default provider from Superset connection schema

* feat: add Superset form handling and connection testing functionality

* Fix the unit test

* Fix Search Index Application spec failure
2025-10-03 14:25:56 +05:30
Akash Verma
5bb2924a6a
Fix #16081 : Add support for SQL Server hierarchyid, geography, and geometry types (#23527) 2025-10-03 11:46:01 +05:30
Akash Verma
4d68fe7a10
feat: Add ML model lineage support (#23494) 2025-10-03 11:38:41 +05:30
Pere Miquel Brull
e0be2820e1
supported timezone and column name fields in SLA form (#23657)
* MINOR - Add SLA fields

* Update generated TypeScript types

* added ui support for the timezone and columnName

* MINOR - Add SLA fields

* Update generated TypeScript types

* change the timezone as the enum and added field placeholder

* display column name on ui

* Fix unit test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-10-02 17:22:37 +02:00
Ram Narayan Balaji
3e3c675760
Improvement: Add reviewers to Metrics (#23662)
* Add reviewers to Metrics

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-10-02 10:45:39 +05:30
Tomas Montiel Prieto
e6edd3c760
Minor - add executedQuery field (#23627)
* add executedQuery field

* Update generated TypeScript types

* Trigger CI

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-10-01 15:10:31 +02:00
Eugenio
dff2b394d5
Fix classification scoring (#23523)
* Add `reason` property to `TagLabel`

This is to understand what score was used for selecting the entity

* Build `TagLabel`s with `reason`

* Increase `PIIProcessor._tolerance`

This is so we correctly filter out low scores from classifiers while still maintaining the normalization that filters out confusing outcomes.

e.g: an output with scores 0.3, 0.7 and 0.75, would initially filter the 0.3 and then discard the other two because they're both relatively high results.

* Make database and DAO changes needed to persist `TagLabel.reason`

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-10-01 12:11:14 +00:00
Adrià Manero
0f1b2aa681
Fixes #23125: Notification template validation and reset (#23536)
* Setup SPI Pattern for seed data path on OM commercial version

* Implement and test Notification Template validation and reset endpoints

* Relocate system-default notification templates

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-10-01 13:30:41 +02:00
Aniket Katkar
ae39c7e68e
Fix(UI): Add service form bugs (#23613)
* Add default filter pattern fields for custom and share point drive connections

* Add translations for additional properties

* Fix the additional properties field showing in the default filters form
Improve the additional properties field input in the form to make it clear with a label

* Update generated TypeScript types

* Empty commit

* refactor(tests): streamline imports in Directory, File, Spreadsheet, and Worksheet class tests

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-10-01 15:18:24 +05:30
Keshav Mohta
6b7262a8ea
Feature: MF4 File Reader (#23308)
* feat: mf4 file reader

* refactor: removed schema_from_data implementation

* test: added tests for mf4 files
2025-10-01 11:19:00 +02:00
Pere Miquel Brull
3842678776
Rename contract consumers and display data on UI (#23626)
* MINOR - Rename contract consumers

* Update generated TypeScript types

* supported contract security data display ui with test

* fix the row_filter data being send on API if they don't posses data

* fix the playwright test because of breaking of adding new rule

* reverted the commented code

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-10-01 10:53:13 +02:00
Pere Miquel Brull
375e001dd9
MINOR - Fix S3 logging from ingestion pipelines (#23590)
* MINOR - Fix S3 logging from ingestion pipelines

* Update generated TypeScript types

* config

* update s3 configurations for streamable logs

* Update generated TypeScript types

* update s3 configurations for streamable logs

* update s3 configurations for streamable logs

* update s3 configurations for streamable logs

* SSE off by default

* Update log retrieval to use s3 if ingestion runner has streamable logs enabled

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2025-10-01 09:44:17 +02:00
Sriharsha Chintalapani
18677afd39
Add support for Tags customizable rules, capturing feedback (#23289)
* Add support for translations in multi lang

* Add Tag Feedback System

* Update generated TypeScript types

* Fix typing issues and add tests to reocgnizer factory

* Updated `TagResourceTest.assertFieldChange` to fix broken test

This is because change description values had been serialized into strings and for some reason the keys ended up in a different order. So instead of performing String comparison, we do Json comparisons

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eugenio Doñaque <eugenio.donaque@getcollate.io>
2025-09-30 07:17:18 +02:00
Sriharsha Chintalapani
bb1395fc72
Implement Modern Fluent API Pattern for OpenMetadata Java Client (#23239)
* Implement Modern Fluent API Pattern for OpenMetadata Java Client

* Add Lineage, Bulk, Search static methods

* Add all API support for Java & Python SDKs

* Add Python SDKs and mock tests

* Add Fluent APIs for sdks

* Add Fluent APIs for sdks

* Add Fluent APIs for sdks, support async import/export

* Remove unnecessary scripts

* fix py checkstyle

* fix tests with new plural form sdks

* Fix tests

* remove examples from python sdk

* remove examples from python sdk

* Fix type check

* Fix pyformat check

* Fix pyformat check

* fix python integration tests

* fix pycheck and pytests

* fix search api pycheck

* fix pycheck

* fix pycheck

* fix pycheck

* Fix test_sdk_integration

* Improvements to SDK

* Remove SDK coverage for Python 3.9

* Remove SDK coverage for Python 3.9

* Remove SDK coverage for Python 3.9
2025-09-29 16:07:02 -07:00
Keshav Mohta
4528c0c1c4
Fixes #23416: Option To Opt Out of BigQuery Policy Tags Ingestion (#23532)
* fix: added includePolicyTags flag

* feat: added includePolicyTags
2025-09-29 18:24:10 +05:30
Ajith Prasad
ed71f3629a
Field level validation in SSO configuration (#23562)
* Field level validation in SSO configuration

* Update generated TypeScript types

* Ldap field error response

* add form field level error validations

* field level error highlight style fix

* fix array field level error highlight issue

* minor style fix for array fields

* resolve merge conflict

* error highlight for text area fields

* fix test

* Fixed tests and improvement

* added unit test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
2025-09-29 10:54:51 +05:30
Aniket Katkar
acc79aab1e
Chore(UI): Drive service bug fixes and unit tests (#23554)
* feat: Add file type, extension, and version fields to file details

- Updated  in  to include , , and .
- Introduced new test file  to cover the functionality of the .
- Enhanced  to export the  class for testing purposes.
- Created  to implement tests for the  functionality.
- Updated  to include  in the default fields for worksheets.

* feat: Add default filter patterns for directories, files, spreadsheets, and worksheets in Drive service

* fix: Add margin to advanced properties collapse for better spacing

* Update generated TypeScript types

* Add tests for new pages
refactor: Move interface definitions for detail page tabs to their respective utils files to fix circular dependency
Add unit tests for SpreadsheetDetailsUtils and WorksheetDetailsUtils

- Created SpreadsheetDetailsUtils.test.tsx to test the functionality of the SpreadsheetDetailsUtils module, including tab rendering and widget retrieval.
- Created WorksheetDetailsUtils.test.tsx to test the functionality of the WorksheetDetailsUtils module, covering tab rendering and widget handling.
- Implemented various test cases to ensure correct behavior for different scenarios, including edge cases for both modules.
- Mocked necessary components and utilities to isolate tests and ensure reliable results.

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-29 09:42:16 +05:30
Shrushti Polekar
134ab5d538
fix(ui): SSO feedbacks and improvements (#23553)
* fix dropdown issue

* update breadcrumb icon

* address styling feedbacks

* fix error taost  content panel resize issue

* update doc stylings

* minor style updates

* update doc stylings

* refactor code to avoid global style

* update delete scim modal

* minor style fix

* minor fix

* fix select dropdown styles

* refactor code to remove !important

* improve login experience

* Update generated TypeScript types

* fix unit test

* update scroll behaviour

* remove eslint issues

* remove old icons

* address pr comment

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-27 21:19:21 +05:30
Ram Narayan Balaji
532f3e5e4e
Undo entity status migration, show entityStatus by default (#23511)
* Revert Entity Status Migrations as we already have the default value

* Batch Migrations and by default show entityStatus field in the response

* Setting default value of enum in the status.json and fixing test cases for new default enum UNPROCESSED

* Update generated TypeScript types

* DataContract by default has Unprocessed status, Overriden method in the GlossaryTermRepository

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-24 18:38:03 +05:30
Aniket Katkar
c6c9eb26e2
Feat: Add drive service (#23018)
* Add the drive service settings option

* Fix the ci error

* Add Directory Details page

* Add Explore page support for new drive assets

* Add Directory Children table

* Update page.json

* Add File details page

* Add spreadsheet details page

* Add Worksheet details page

* Update explore tree name

* Fix the search suggestions not showing for new assets

* Add new assets in missing setting options

* Update service util class base for drive service

* Add version support for directory

* Add version pages for File, Spreadsheet and Worksheet

* Add FQN for worksheet columns

* Add columns table for the worksheet

* Fix the Spreadsheet, Directory and Worksheet version pages not showing children.

* Fix the domain not updating for drive assets

* Fix announcement not working for drive assets

* Add playwright for Directory asset

* Fix the index mappings and failing playwright test for directory

* Add google drive service icon

* Fix the delete and restore functionality for drive assets

* Fix patch tags for worksheet and add tests

* Fix failures

* search indices

* Update service details page for drive service

* Add playwright tests for remaining drive assets

* Fix listing APIs to return when root=true mentioned

* Update API to fetch drive service assets and fix playwright for Worksheet service

* Fix the index mappings

* Add the new fields for advanced search

* Add the playwright tests for drive assets

* fix: drive service test connection

* fix: drive service test connection

* Fix the ServiceDetailsPage tests

* Fix the playwright tests

* Fix the search settings issue for drive assets

* add missing icons, fix search index issue for worksheet, resolve the sonar issues

* Fix the entity permission spec

* Fix FileResourceTest to create GoogleDrive instead of SharePoint to match the type creation in the test

* Work on comments

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Keshav Mohta <keshavmohta09@gmail.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
2025-09-24 13:37:31 +05:30
Chirag Madlani
8deab28071
feat(ui): support impact analysis for lineage (#23231)
* feat(ui): support impact analysis for lineage

* WIP filters

* Add Node Depth and fix downstream and upstream counts

* integrate search / quick filters with api

* fix api calls

* Fix Listing and remove unnecessary call for root entity

* Fix OS

* support column level lineage for impact analysis

* update

* Add Pagination Apis and Node Depth apis

* Fix Looping infinitely

* Add Validation

* Reduce Response size for lineage

* Add node depth upto

* Add Edge Information

* incorporate new apis

* push queryFilter to fetchNodes

* query_filter -> queryFilter

* add export for impact analysis

* update pagination

* fix query_filter param

* Fix issue on upstream lineage result due to filters

* Fix Filters issue

* Update generated TypeScript types

* update unit tests

* fix tests

* address comments

* support impact analysis for all the tabs

* update icon for column customization

* Fix breaking tests

* update test

* update translation

* Fix Wrong Match for UPSTREAM

* clean console

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-24 12:53:20 +05:30
Mayur Singal
933802a354
MINOR: Support metabase API Key Auth (#23436) 2025-09-23 22:01:10 +05:30
Keshav Mohta
cb26c91442
Revert "Fixes #23356: Databricks OAuth & Azure AD Auth (#23482)" (#23530)
This reverts commit f1afe8f5f114ee58090168fd7ae5d66b38a01ab0.
2025-09-23 17:44:16 +02:00
Keshav Mohta
f1afe8f5f1
Fixes #23356: Databricks OAuth & Azure AD Auth (#23482)
* feat: databricks oauth and azure ad auth setup

* refactor: add auth type changes in databricks.md

* fix: test after oauth changes

* refactor: unity catalog connection to databricks connection code
2025-09-23 15:22:50 +05:30
Shrushti Polekar
eebd3bb661
Feat(ui): SSO configuration from UI and Scim bot token generate revoke functionality and Group Mapping in settings page (#21839)
* added scim bot generate revoke token functionality on settings page

* update global setting constant

* minor code refactor

* WIP: SSO Configuration from UI

* Update generated TypeScript types

* Copilot suggestions addressed

* Test Endpoint added

* Added update security configuration test

* added sso configuration rjsf form

* minor  fixes

* Update generated TypeScript types

* update scim component

* Fix Merge Conflicts

* Remove extra comma

* update sso form for oidc client providers

* SAML configuration fixes

* Update generated TypeScript types

* Update generated TypeScript types

* add support for selective schema generation by file name

* fix acces token card issue and array fields ui issue

* Saml registration fix

* Scim configuration saved

* refactor code

* scim config issues fixed

* fix enableSelfSignup

* scim enabling from settings

* Test validation enpoint changes

* fix ldap issues

* minor refactor

* minor updates

* update settings call for scim

* update bot details in scim card

* minor style fixes

* Adding enhanced validators

* update confidential client configurations

* Auth0 validator

* update jwt for saml

* Saml validator changes

* Okta validator fix

* fix saml certificate formatting and select widgets

* remove client type from ui for ldap and saml

* Okta validation rectified

* Google validator improved

* update select component

* fix error messages

* added unit test

* Azure validator rectified

* Validation issues solved for auth0

* add oidc prefix for oidc client fields

* fix type issues

* style fixes for scim token card

* Update cognito validator

* refactor file names

* added e2e tests

* minor fixes

* added content for docs in md files

* implement  new design

* form style updates

* fix doc styles and provider selection styles

* fix provider selection style

* update local file

* group client fields together

* added provider name

* overview tab changes

* update form styles for configure tab

* update toggle styles

* update breadcrumb texts and provider name in text

* minor fix in localization

* minor fix for basic provider

* update breadcrumb styles and minor local file update

* minor fixes

* minor fix for local file

* fix styles for configured form

* minor fixes in styles

* fix redirect issue of my-data for non oidc sso

* fix build issue and refactor code

* update saml and ldap icons

* minor updates in icon

* typo fix

* breadcrumb and style fixes

* fix test

* update patch call

* minor fixes for access token card and code refactor

* refactor code to remove unused files

* added ldap and custom oidc doc content

* fix overview tab issue for scim and enable sso

* Patch issue fixed for authentication Configuration

* Implement unified authentication endpoints across all providers

* Memory leak issue solved

* Mcp md file

* apply spotless

* Added complete flow test for oidc

* refactor css files

* minor style updates

* cancel form functionality

* update placeholders

* update cancel modal

* minor update for saving existing config

* fix minor css issue

* integrate unified api

* remove clientid and callbackurl for confidential client type and added  jwtPrincipalClaimsMapping

* fix breadcrumb issue

* refactor files and folder structure

* minor fix

* refactor code and add disable fields styles and cancel btn functionality for existing sso

* added localization file changes

* Patch operation issue fixed

* Update generated TypeScript types

* update patch call

* patch call update for enable self signup

* remove enable selfsignup from then form since its present in the toggle

* added enable self signup field for non existing sso form

* minor code revert for enabling fields

* Update generated TypeScript types

* Redirect uri, password encoding fixes for basic and ldap

* refactor code and encode password and added /auth/callback in redirecturl

* minor  fixes

* added group mapping

* cancel modal save btn fix and logout fix

* minor fix for save btn functionality for cancel modal

* minor update in localization file

* address pr comments for ssoconfigform and css file names

* address pr comments for sso doc panel

* address pr comments around provider selector

* fix breadcrumb

* update localization file and remove unwanted files

* address pr comments related to basic auth provider

* extracted proxy code into separate func and added dev mode check

* remove unnecessary validate call happening during patch calls

* address pr comment related to tab

* update sso icon

* minor fix

* address parseSchema related pr comment

* address pr comment related to config fields constants

* added default value for callback url

* added default value for response type and clientAuthenticationMethod

* minor fix for callback url

* minor style fix

* Changed response to json for basic and ldap login

* revert unified api changes done for basic login

* made admin principal as required

* fix playwright for llogin test failure

* fix failing unit test

* fix login e2e test

* refresh call

* fix failing unit test

* fix unit test

* fix test

* remove setRefreshToken

* added default values for google

* minor style fixes

* minor text update

* Google credential validation, scope validation

* Discovery uri validation added with Auth0

* Discovery uri check added for okta

* minor style fixes

* minor checkbox fix

* minor required field fix

* minro fix

* AWS cognito validator

* google api depdendency added

* Update generated TypeScript types

* minor fixes

* minor fix for array fields cursor style issue

* Modified saml validator

* Admin principal respected while logging in

* Added tests for validators

* callback url update

* custom oidc validator

* added default values for saml

* custom oidc fix

* Saml validator issue solved

* Custom oidc publickeyurl validation

* validation for prompt

* test fix

* fix test

* TC fixed

* mvn spotless apply

* Addressed review comments

* Static method issue in Security COnfiguration manager fixed

---------

Co-authored-by: aji-aju <ajithprasad770@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Ajith Prasad <37380177+aji-aju@users.noreply.github.com>
2025-09-23 13:23:46 +05:30
IceS2
a630ca2be6
Fixes #23330: Implement Fetch DQ Test Cases by FollowedBy (#23461)
* Implement Fetch DQ Test Cases by FollowedBy

* Update generated TypeScript types

* Fix wrong conflict resolution

* Refactored into Optional

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-23 01:08:49 +05:30
Ram Narayan Balaji
78d71723a0
Add createdBy, createdAt for dataContracts (#23427)
* Add createdBy, createdAt for dataContracts

* Update generated TypeScript types

* Preserve createdBy, createdAt, createdBy marked as string

* Set createdBy, createdAt during prepare method

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-09-22 10:49:05 +02:00
Ram Narayan Balaji
edebee9b73
Add entityStatus and reviewers to testCase, reviewers for testSuite (#23340)
* Add entityStatus and reviewers to testCase

* Update generated TypeScript types

* Filtering fields based on parent for testCase

* Reviewers for Test Suite, Inherit Reviewers from all test suites for this test case

* Update generated TypeScript types

* getFiltered Fields centralised in the EntityUtil

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-16 18:20:38 +00:00
Adrià Manero
840f7e2a68
Setup NotificationTemplate CRUD (#23293) 2025-09-16 15:21:11 +02:00
sonika-shah
cc18ba1554
feat: Add configurable filters aggregation for tag source segregation in search (#23363)
* feat: Add configurable filters aggregation for tag source segregation in search

* Update generated TypeScript types

* use script based aggregation

* use script based aggregation

* Update generated TypeScript types

* update aggregations with script

* add migrations to fetch updated searchSettings

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-09-16 15:16:06 +05:30
Karan Hotchandani
7672f85592
chore(ui): move from terser to esbuild (#23410)
* move from terser to esbuild

* Update EntityLink.g4

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2025-09-16 14:43:06 +05:30
Teddy
dad07bc7fa
ISSUE #23281 - changed passed/failed[Rows] to Long to handle 64-bits numbers (#23325)
* fix: changed passed/failed[Rows] to Long to handle 64-bits numbers

* fix: failing test case
2025-09-16 08:38:33 +02:00
Ayush Shah
e933089158
feat: extend ENTITY_TYPE with additional entity types and fields (#23403) 2025-09-16 08:27:43 +05:30
Sriharsha Chintalapani
cf7931ee3b
Add logging endpoint into S3 (#22533)
* Add logging endpoint into S3

* Update generated TypeScript types

* Stream Ingestion logs to S3

* Update generated TypeScript types

* Address comments

* Update generated TypeScript types

* create logs mixin, use clients to stream logs

* centralize logs sending into mixin

* use StreamableLogHandlerManager instead global handler

* improve condition

* remove example workflow file

* formatting changes

* fix tests and format

* tests, checkstyle fix

* minor changes

* reformat code

* tests fix

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: harshsoni2024 <harshsoni2024@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-09-15 07:22:25 -07:00
Keshav Mohta
1f379a8697
fix: added depth in json and pass in metadata entry (#23332) 2025-09-12 12:31:20 +05:30
Ram Narayan Balaji
9fd34c8f89
Feat #20586 Implementation of Custom Metrics Measurement Units (#22876)
* Initial Implementation of Custom Metrics Measurement Units

* Update generated TypeScript types

* Removed Regex patterns and length validations as they are not needed

* Add a new column with index for custom units

* Remove comments in the sql

* update ui and add playwright

* fix metric selector

* fix tests

* address feedbacks

* remove unused field

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-09-11 15:14:38 +05:30
Mayur Singal
d705fffc1d
Fix #1968: Query Runner Schema (#23077) 2025-09-11 10:41:11 +05:30
Teddy
f3cb001d2b
ISSUE #2033-C - Support For DBX Exporter + Minor Fix to Status (#23313)
* feat: added config support for databricks

* fix: allow incrementing record count directly without storing element

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-10 12:04:46 +02:00
Suman Maharana
39cb165164
Feat: show dbt project name (#23044)
* Feat: show dbt project name

* Update generated TypeScript types

* added dbtSourceProject in data asset header properties

* Added tests

* Addressed comments

* Update generated TypeScript types

* move from dataAssetHeader to the dbt tab itself

* added unit test for added code

* test name change

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-09-10 11:23:28 +02:00
IceS2
8177e529bc
FIXES #23220: Add cardinality metric for string and enum (#23052)
* Implement Cardinality Metric for String and Enum

* Add Unit Tests

* Update generated TypeScript types

* Update ingestion/src/metadata/profiler/metrics/hybrid/cardinality_distribution.py

Co-authored-by: Teddy <teddy.crepineau@gmail.com>

* Fix CTE to simplify it to work with sqlite

* Fix CTE to simplify it to work with sqlite

* Update generated TypeScript types

* Update generated TypeScript types

* Add 'cardinalityDistribution' metric to profiler configuration

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2025-09-09 16:38:53 +02:00
Pere Miquel Brull
4483e183cb
MINOR - Update contract security schema (#23274)
* MINOR - Update contract security schema

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-08 10:42:34 +02:00
Teddy
568baab426
ISSUE #2031 - Add App Runner to Application Entity (#23259)
* feat: added exporter app config

* refactor: added entityprofile resource & added backward compatibility to existing API

* feat: added tests to get_profile_data_by_type

* feat: remove non supported event types

* chore: added migrations to 1.9.7

* chore: added application creation readme

* feat: implemented backend support for agent runner for external applications

* chore: move migrations to 1.9.8

* Update generated TypeScript types

* fix: added setIngestionRunner method to serviceEntityInterface interface

* Update generated TypeScript types

* fix: remove service instanceof MetadataService in deploy endpoint

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-08 08:04:50 +02:00
sonika-shah
40ccc35165
Add entityType.keyword field with lowercase normalizer for case-insen… (#23147)
* Add entityType.keyword field with lowercase normalizer for case-insensitive search

* fix(UI): Updated Enum For Search (#23163)

* updated enum

* fixed flaky entitysummerypanel

* fixed failing tests

* fixed failing playwright test

* fixed failing tests

* fixed wait for response for playwright

* fixed api request for playwright

* fixed api call for playwright

* fixed explore tree

* addressed comments

* made filter lowercase to match with dropdown

* fixed failing playwright tests

* fix Users.spec

---------

Co-authored-by: Dhruv Parmar <83108871+dhruvjsx@users.noreply.github.com>
Co-authored-by: Pranita Fulsundar <pfulsundar8@gmail.com>
2025-09-08 09:18:18 +05:30
Ram Narayan Balaji
c24701b797
Added entityStatus and reviewers property in ru index mappings (#23286) 2025-09-08 09:17:44 +05:30
Teddy
1ef191a2aa
ISSUE #1534 - Profiler Refactor for Metadata Extraction Application (#23200)
* feat: added exporter app config

* refactor: added entityprofile resource & added backward compatibility to existing API

* feat: added tests to get_profile_data_by_type

* feat: remove non supported event types

* chore: added migrations to 1.9.7

* chore: added application creation readme

* chore: move migrations to 1.9.8

* fix: failing java test

* style: ran java linting
2025-09-05 13:07:04 +02:00
Adrià Manero
571d7ee2eb
Allow Alerts to notify Lineage downstream entities stakeholders (#23218)
* Setup schema and core lineage traversal service for retrieving downstream entities

* Update getTargetsForAlert signature to use SubscriptionDestination

* Modify alert target finder method to support lineage entities analysis

* Rename utility class and method to explore downstream entities of the lineage graph

* Apply style

* Prevent entering on recursion when maxDepth is lower than 0

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-09-05 12:49:37 +02:00