16 Commits

Author SHA1 Message Date
Mohit Yadav
e9ab9dc985
Lineage Improvements (#19639)
* - typo update

* - Update lineage model and api

(cherry picked from commit c5cce2b0c7c1be7ba654bfc2677da0d05215a895)

* - Adding patch

* - spotless fix

* Add Migration For createdAt , createdBy, updatedAt and updatedBy

* Moved classes from Elastic Client to Lineage Graph Builder

* Update to use correct depth

* remove toCondition

* Fix Downstream Lineage

* moved not so cool things here and there

* spotless-failures

* call with direction

* add upstream depth with + 1 for 0th node

* Fix Exports

* Fix Exports from and size

* parse lineage in ui

* Add per entity pagination

* Added per entity pagination

* Fix Async Export

* Fix Java Merges

* Fix merge conflict

* Fix missing upstream nodes

* lineage ui initial commit

* Fix Pagination for upstream

* Fix Values

* fix load more loading

* cleanup

* fix loading of nodes

* fix load more

* Fix Direction Values

* update expand collapse logic

* Fix Deletion Issue

* Fix Pipeline Lineage

* show pipeline as a node

* Use Fqn Hash to find entities is es for lineage

* update lineage

* Add Lineage for Pipeline to look for as edge and node

* Layer and size conditions for upstream and downstream

* Add include source fields in lineage request

* redesign controls component

* add lineage fixes

* fix tests

* Fix node

* Fix Some Failing Test on Lineage

* Java Spotless Fix and added missing relation check on migration

* Missing conflict changes

* fix conflicts

* added missing keys

* update tests

* Add Service Information in service indexes

* Add Domain Lineage

* Add domain lineages

* fix tests

* Fix Metric Entity Service Error

* Maintain Asset Count for entities adding service and domain lineage

* Update to Doc Id and add details to search

* Cleanup Service and Domain automatically

* fix types

* Update review comments

* fix icon sizes

* Update Services Index wit upstream lineage

* add service view

* Fix Service Name missingFix Service Name resolution in LineageRepository

Update the method to correctly retrieve the service name for the "fromEntity" by specifying the FIELD_SERVICE path. This resolves an issue where the service name was not properly populated.

* - Add docUniqueId to lineage data and update related queries
- Added lenient Object Mapper for search index to entities mapping

* - Fix LineageResourceTest failing due to changes on Lineage Storage

* - Update Lineage Details

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-03-10 10:09:29 +05:30
Ashish Gupta
3924e58073
MINOR: fix the lineage and DataAssetsWidget flaky playwright test failure (#19560)
* fix the lineage flaky test failure

* fix the DataAssetsWidget falky test failure

* fix falky test around the pipeline modal and teams
2025-01-29 11:38:01 +05:30
Shailesh Parmar
7d7bc5b48b
playwright: fixed aut failure in playwright (#18958)
* playwright: fixed aut failure in playwright

* Temporarily increased the timeout for lineage tests which timeout often

* Fix the flaky tests

* removing linage changes

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-12-09 18:03:13 +05:30
Aniket Katkar
f6b4434e4a
Improvements: Playwright tests (#18849)
* Resolve all the type errors in playwright code

* Fix the playwright tests

* Improve the ingestion test connection logic and autoClassification

* Fix flaky tests

* SearchIndexApplication fix

* Fix the flaky tests

* flaky test fixes and trace on for debugging

* Fix the flaky tests

* update the trace config for data insights setup

* Fix flaky tests

* Fix the flaky tests
2024-12-05 17:53:14 +05:30
Karan Hotchandani
fe661a2f49
Change lineage positioning algorithm (#18897)
* use elk algorithm to position nodes

* change positioning

* fix spacing

* do not reset zoom value

* minor pw fix

* force click on lineage edge
2024-12-04 21:27:43 +05:30
Sriharsha Chintalapani
9484d838a4
Minor: fix Lineage export when there is no column / pipeline edge (#18737)
* Minor: fix Lineage export when there is no column / pipeline edge

* add test for lineage export (#18709)

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* add lineage export tests

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sweta Agarwalla <105535990+sweta1308@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2024-11-22 08:21:19 -08:00
Teddy
429e48aa3a
#18677 - Added Lineage Field back to SearchLineage (#18687)
* fix: add lineage field back to searchLineage

* add lineage tests

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
2024-11-20 11:13:09 +01:00
Karan Hotchandani
7cba2a7f51
fix lineage tests (#18618) 2024-11-13 13:29:51 +05:30
Karan Hotchandani
3dcbeb469f
App settings - lineage config (#18522)
* add lineage settings page

* update locales

* minor ui changes

* add playwright for lineage config
2024-11-06 10:26:18 +00:00
Karan Hotchandani
47cde167b4
GEN-712: show column lineage function in edge drawer (#18032)
* show column lineage function

* add e2e tests
2024-09-30 15:00:47 +05:30
Karan Hotchandani
58ed12cf47
fix lineage playwright (#17889)
* fix lineage playwright

* remove in operator usage and use lodash get utility

* fix metric playwright

* fix lineage pw

* update drag and drop

* fix flakiness

* minor sonar fix

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-09-30 10:54:39 +05:30
Ashish Gupta
49323ed5bc
fix failing playwright test (#17881) 2024-09-17 16:01:58 +05:30
Sriharsha Chintalapani
c34545adeb
Issue-15768: Support Metric Entity (#17680)
* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Fix tests

* Fix tests

* Fix tests

* Minor: Fix tests

* ui: add metricsAPI rest utils

* ui: metric list page part 1

* feat: Add metric translations for multiple languages

* chore: Add "metric" field to SearchIndexingApplication schema

* ui: add create metric page

* ui: metric details page patch 1

* ui: add custom property and lineage support for metric entity

* ui: add expression component

* ui: add metric summary component

* chore: Update tab labels in MetricDetails and MetricVersion components

* ui: show other info like metric type, granularity, etc

* feat: Add support for metric entity in search dropdown

* feat: Rename custom property to Metric in MetricEntity.md

* feat: Add OwnerLabel component to MetricListPage

* Fix expression field in Metric

* chore: update expression to metricExpression

* ui: add metric header component with edit option

* Add metric to SearchIndexApp

* chore: Update expression to metricExpression

* ui: allow metric expression edit

* ui: update metric icon

* minor improvements

* Fix lineage indexing for Metric

* Update GlobalSettingsClassBase.ts to use MetricIcon for metrics in the global settings menu

* Fix error handling in MetricListPage component

* add related metrics

* minor improvements

* Fix relatedTerms patch

* Fix relatedTerms validation

* Add Boolean for deleted

* filter active entity from related metric list

* playwrite e2e part 1

* Refactor MetricSummary component to include RelatedMetrics in the summary panel

* test: add playwright test for metric special cases

* Add 'Metrics' to Explore Tree

* test: add e2e for add metric page

* test: add test for metric listing page content

* Add Boolean for deleted, remove deleted from suggests

* Refactor LineageProvider to handle deleted flag properly

* add playwright for metric listing

* fix test

* Add colored metric icon and update its usage in GlobalSettingsClassBase

* Fixed py_test test_ometa_endpoint for metric

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: SumanMaharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-09-16 11:36:31 +05:30
Sachin Chaurasiya
d3cbd5e428
minor: add column level lineage support for api endpoint (#17334) 2024-08-10 14:31:14 +05:30
Karan Hotchandani
fe74b27033
Allow adding pipeline between all entities in lineage (#16960)
* fix tier information

* add tests

* fix tests

* update test as slow
2024-07-09 15:33:28 +05:30
Karan Hotchandani
607440f804
Lineage improvements (#16785)
* lineage improvements

* show deleted icon on node

* fix reposition of nodes

* add lineage playwright

* fix tests

* add playwright

* fix flaky tests

* fix repositioning

* feedbacks
2024-06-27 15:32:58 +05:30