7194 Commits

Author SHA1 Message Date
Onkar Ravgan
8fac9d8976
Added Powerbi Lineage across workspaces and datamodel projects (#12052)
* powerbi lin multiple workspace

* Addressed review comments
2023-06-21 17:16:09 +02:00
Onkar Ravgan
0cfd8d29c5
Added dbt test_metadata check (#12067) 2023-06-21 12:17:26 +02:00
Ayush Shah
9fa7b78d87
Issue 11686: Add IncludeOwner + scheduleInterval in Airflow (#12005) 2023-06-21 14:18:45 +05:30
Onkar Ravgan
47514f6c5f
Fixed dbt test col name (#12065) 2023-06-21 10:47:54 +02:00
Chirag Madlani
b9d85bb27f
UI - design update for new UI/UX (#12000)
* chore(ui): revamp table details page
Re-design table details header with schema tab

* misc fixes for ui and tests

* update

* fix edit description

* chore(ui): update the app left panel icons

* chore(ui): update left panel icon stroke color

* fix: explore page layout change

* fix: highlight issues

* updated right panel for explore page

* fix: explore layout changes

* separate out header component

* added activity feed tab

* fix: highlight issue

* fix: highlight card for explore page

* support all widget
fix spacing issue

* fix: show reactions

* fix: update reactions for tasks tab

* fix: make use of appstate to avoid repetitive calls

* fix activity feed layout

* fix: landing page and explore feedbacks

* fix: missing localization

* update activityFeedProvider with drawer

* fix description v1 component icons

* fix: update styles

* minor fix on icons and entity table

* Revamp Tag UI for TableDetailPage

* fix: update css

* Remove color in TagV1 component and minor file improvement

* added frequantly joint table basic structure

* fix: update kpi chart styles

* supported request tags in table detail page

* fix request description redirect issue

* fix: update styles for my data page

* added scroll to frequently joint table

* removing unnaccessary code

* fix: explore page feedbacks

* fix: explore feedbacks

* fixed signup page issue

* chore(ui): only apply the text transform on heading

* chore(ui): update table header styling

* chore(ui): update table text color and border radius

* fix: explore quick filters

* update header for topic and dashboard

* chore: update table body styling

* supported glossary term hierarchy

* updating spacing for all the page

* chore(ui): update descriptionV1 component styling

* chore: update add-chat viewbox

* fix: left sidebar console errors

* update all the entity headers

* chore: update table name to name

* update description component

* clear console errors

* revamp entity topbar manage area

* fix: entity popover similar to explore card

* updated navbar as per new UI mock

* fix: feedbacks

* updated announcement style as per mock

* fix: explore feedbacks

* implemented single box shadow wherever its needed and worked on feedback

* worked on feedbacks

* update feed tab on activityFeedTab component

* updated glossary card, no owner icon

* fix glossary hierarchy issue of repeating same glossary

* fix: update resizable panel pages

* updated navbar, align profiler picture in navbar with other element

* fix: remove page layout component

* fix: add missing layout in rule pages

* fix: jest tests

* fix: jest tests

* replace old activity feed component with new one

* fix: jest unit tests

* initial DQ page setup

* translation sync

* fix: unit tests

* supported subtab route for entities pages

* fix: feedbacks

* fix: update tasks pages

* supported api for activity feeds for entity

* added tabs component in entity table component

* worked on test-suite tab

* added quality page path in left bar

* added action button for test suite

* updated redirect path for data-quality

* revert to old re-direction for add-test suite

* update activity feed task tab

* hide action for activity feed tab

* fix: task action issue on descriptionV1

* supported new ui for data modal detail page

* fix: feedbacks

* fix: add missing localization

* fix: breadcrumbs

* data quality feedbacks

* fix: feedbacks

* fix: add missing localization

* fix: feedbacks

* supported activity feed in container and tags of all entities

* update task relates changes

* remove inline css for entity tag component

* feat: update blue as primary color

* feat: update svgs to use new primary color

* updated DQ page with feedback, and add quality page flow change

* translation-sync

* fix assignee for tasks

* updated api data for test suite pipeline creation

* fix: primary color changes

* fix: feedbacks

* fix count issues

* fix scrolling issue for activity feed

* address feedbacks

* combine open task with heading buttons

* data quality changes

* fix close with comment

* removed delete functionality

* clear CLI error, and changes as per JSON schema

* update scroll issue on details page

* added new badge, and api integration

* chore: update data quality tab page styling

* cleanup

* fix settings page scroll issue

* fix scroll issue on service page

* chore: update summary cards

* update graph color

* DQ test case test as per mock

* support edit and suggest

* integrated API for status update

* translation sync

* fix cypress tests

* integrated API call for test summary

* mydata and following page change in user profile page

* fix tag cypress tests

* updated success % via api data for test suite

* fix user component unit test

* integrated API for test summary

* updated data quality tab as per new mock

* revert commented code fir testcaseForm.tsx

* fix cypress tests

* change user page activity feed with new ui

* chore: fix console errors related to activity feed tab component

* chore: update setting button style

* chore: fix spacing

* added pipeline tab

* chore: update the colum profiler table

* fix: test indicator styling

* fetch test suite details

* fix: lineage redesign

* fix: missing localization

* chore: add support for showing single column profile in page itself

* chore: add column dropdown for column profiler

* fix cypress for entity tags add and remove

* supported more parameter in entity dashboard header

* chore: add default tab for data quality

* removed unneccessary code, and updated testsuite details page with new style

* fixed DQ test

* translation sync

* fix cypress

* added functionality to link test case with logical test suite

* chore: remove fixed right from status column in column profile tab

* fix: purple color issues

* fix the count issue in user profile page and breaking of task page

* fix: feedbacks

* fix: feedbacks

* ui improvements

* fix cypress for owner and tier

* change announcement card color and deleted old assest to it

* remove old activity feed files

* fix unit test

* fix various cypress

* fix cypress failues

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-06-21 12:59:20 +05:30
Pere Miquel Brull
8a9566aa2f
Update dbt docs for S3 (#12059) 2023-06-21 08:36:29 +02:00
Pere Miquel Brull
7f39cc105f
Fix Airflow docs (#12009) 2023-06-21 08:36:06 +02:00
Pere Miquel Brull
35cca0e178
Add automations workflow truncate migration (#12007) 2023-06-21 08:35:50 +02:00
Pere Miquel Brull
bfa37df298
Add platform in Pipeline Service status msg (#12041)
* Add platform in Pipeline Service status msg

* chore(ui): show message based on platform

* update the styling

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-21 07:21:19 +02:00
Pere Miquel Brull
3c430ca722
Fix #11966 - Add delete secrets options for managed SM (#12051)
* Add delete secrets options for managed SM

* Align tests with mysql auth changes
2023-06-21 06:59:02 +02:00
Chirag Madlani
ef578ecb40
feat(ui): support disable option for tags and classification (#10925)
* feat(ui): support disable option for tags and classification

* backend changes for PATCH API to work for updating disabled flag in classifications and tags

* made UI changes to provide enable/disable option for classifications and tags

* localization changes for other languages

* Improved tags disabling functionality

* changes to disable the system tags
fixed permission checks while editing tags

* localization change for other languages

* fixed unit tests and minor improvements

* fixed failing cypress tests

* disabled the edit actions for the disabled classifications

* localization changes for other languages

* changes to use disabled badge

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2023-06-21 00:13:05 +05:30
Mohit Yadav
131967a53a
Bug fixes for Activity Feed + Restrict digits to 2' place (#12050)
* Fix Activity Feed Ownership Exception, + field specific parsing issue in case of different entities

* Restrict Percent Change to 2 Digits
2023-06-20 23:56:48 +05:30
Pere Miquel Brull
e6f9150cdf
Revert "Add delete secrets options for managed SM (#11972)" (#12049)
This reverts commit 79405080b3427db2ffe1752c6500eca6cbd3b28c.
2023-06-20 15:46:53 +02:00
Pere Miquel Brull
79405080b3
Add delete secrets options for managed SM (#11972) 2023-06-20 15:28:03 +02:00
Mayur Singal
c3cec54be9
Fix for #11807 Part 2: Add SourceUrl for table entity (#12013)
* Fix #11807 Part 2: Add SourceUrl for table entity

* address review comments: centralize sourceurl

* remove qlick

* pytest fix

* fix typo
2023-06-20 11:46:45 +02:00
Pere Miquel Brull
8751427679
Add project to Data Models JSON Schema & Implement in Looker (#12030)
* Add project to dashboards and models

* Add project to dashboards and models

* Update openmetadata-spec/src/main/resources/json/schema/entity/data/dashboardDataModel.json

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

* Update openmetadata-spec/src/main/resources/json/schema/api/data/createDashboardDataModel.json

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

* Fix tests

---------

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-06-20 11:45:29 +02:00
Mayur Singal
2738310887
Fix #12012: Fix clickhouse usage schema_name error (#12032) 2023-06-20 11:41:28 +05:30
Ayush Shah
83e9b6c310
Fixes 10395: Validation of yaml workflow configs (#11985) 2023-06-20 11:20:59 +05:30
Suresh Srinivas
e09537eeeb
Fixes #11883 - A user can resolve a task created by him (#11998)
* Fixes #11883 - A user can resolve a task created by him

* Test failure fixes
2023-06-19 16:29:05 -07:00
Mohit Yadav
7adf1ec2b5
Reindex improvement (#12036)
* Reindex added for TestCase

* Add Mapping Api endpoint
2023-06-19 23:21:29 +05:30
Teddy
73808bf29b
Update dq ingestion workflow (#12037)
* fix: fetch service from sourceConfig.entityFullyQualifiedName

* fix: python linting
2023-06-19 18:01:26 +02:00
Onkar Ravgan
f07c421264
Removed Empty Description Assignment to entities and added database name logic to tableau (#12031) 2023-06-19 19:19:42 +05:30
Teddy
76f5d3d571
Fixes #11994 - Update dbt and GE integration with new DQ flow (#12018)
* feat: updated GE integration to match new test workflow

* feat: updated unit tests to match new signature

* feat: added GE integration tests

* feat: ran python linting

* feat: updated dbt ingestion to match new TestSuite workflow

* feat: ran python linting

* feat: remove testSuite from Elasticsearch event test case update

* feat: ran java linting
2023-06-19 15:05:51 +02:00
Sriharsha Chintalapani
9dbaabad44
Adding MD5 hash to fullyQualifiedName and names to simplify DB indexes for lookups and increasing the size of FQN (#11960)
* Fix fqn hash

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* merge commits

* merge commits

* merge commits

* Fix glossary import/export

* Fix BotResource Tests

* Fix BotResource Tests

* Fix Glossary Term tests

* Fix Glossary Term tests

* Fix Glossary Import/Export tests

* All backend tests are fixed.

* merge with main

* Fix tests and revert local changes

* Fix tests and revert local changes

* Fix NullPointerException for Glossary and Query

* Fix Tests

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-06-19 15:43:05 +05:30
Pere Miquel Brull
4a8554c313
Fix Redshift performance and memory issues for ingestion-base (#12002)
* base op

* Update redshift deps and ingestion-base image

* Remove vim

* Remove vim
2023-06-19 12:09:09 +02:00
Pere Miquel Brull
7d39584bc9
Fix Airflow callback import (#12008) 2023-06-19 11:39:23 +02:00
Onkar Ravgan
d6d721225d
Fixed mark deleted all dbs (#11996) 2023-06-19 12:59:13 +05:30
Mayur Singal
8ecd88fd1b
Fix #11312: Deprecate ES Reindex on Ingestion and use reindex APIs instead (#11988) 2023-06-19 12:45:18 +05:30
Sriharsha Chintalapani
146c56abab
Fix #12014: Register OpenMetadata as a service when the server starts for the first time (#12015) 2023-06-16 16:39:55 -07:00
Teddy
bc76515a6b
fix: release connection from engine pool (#12006) 2023-06-16 16:49:55 +02:00
Onkar Ravgan
3389fcc2a5
removed readwrite docs (#12004) 2023-06-16 15:56:58 +02:00
Sriharsha Chintalapani
817c5f3d4a
Add get tableProfilerConfig test (#9324) 2023-06-16 16:02:38 +05:30
Mohit Yadav
41c49b9de8
Data Insight Report Job Bug Fixes (#11975)
* Data Insight Report Job Bug Fixes

* ui: add enabled field

* test case fix

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-16 15:40:44 +05:30
Ayush Shah
f80eaf3a26
Fixes 11068: mysql & postgres iam auth (#11937) 2023-06-16 13:18:12 +05:30
Teddy
8ad8001da1
skip dbt cli e2e (#11999)
* skip dbt cli e2e

* ran linting
2023-06-16 09:20:35 +02:00
Sachin Chaurasiya
abe8d5c85e
chore(ui): move connection option and arguments under advanced config (#11910)
* chore(ui): move connection option and arguments under advanced config

* chore: show advanced properties inside advanced config

* fix: typescript console error.
2023-06-16 12:50:05 +05:30
Sachin Chaurasiya
a876889924
chore(ui): improve test connection workflow delete flow (#11912)
* chore(ui): improve test connection workflow delete flow

* delete workflow on component unmount

* add unit test

* chore: simplify the condition

* chore: refactor delete workflow method

* fix: alignment issue on server connection tab

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-06-16 12:49:28 +05:30
Teddy
3f01ee938f
Fixes #11895 - Add Indexing and Search logic for TestCases (#11989)
* feat: added logic to delete logical test suite + added check to prevent adding existing testCases to executable test suite

* feat: added elasticsearch index logic for testCases

* feat: added deletion logic from index logic when deleting test suites

* feat: added test case index search to  endpoint

* feat: add executable/logical filter in list testSuite + filterOut tables without tests in Table resource

* feat: added summary field to testSuite

* feat: added executionSummary endpoint for test cases

* feat: removed tick marks around timestamp

* feat: addressed test failures

* feat: ran python linting

* feat: add limit to fetch all tables in TableResource testSuite test

* feat: fix conflict

* feat: ran java checkstyle

* feat: fixed mongo linting + disabled mongo failing tests

* feat: removed mongo test skip

* feat: removed unsued pytest import
2023-06-15 21:27:54 +02:00
Teddy
db59207ffe
temporarly disable dbt test while working on #11895 (#11997) 2023-06-15 20:27:44 +02:00
Onkar Ravgan
d08c928801
Added project property to dashboards (#11986)
* Added projects to dashboards

* Added powerbi proj

* merge conflicts after source url

* fixed mongo pytest
2023-06-15 21:23:43 +05:30
Mayur Singal
82a0222257
SourceUrl changes for dashboard, pipeline & chart entities (#11991) 2023-06-15 14:44:48 +05:30
Pere Miquel Brull
75aa3d5fa1
Fix #10542 & #10830 - Mask Sample Data PII & Column Metrics (#11919)
* Mask Sample Data PII

* Mask column profile

* Mask Test Cases

* Mask Test Cases

* Add tests

* Format

* Format

* Fix test case resource masking
2023-06-15 09:13:57 +02:00
Dương Phan Nam Hưng
105dc9d48d
Update sdk/python/ingestion/lineage.md docs (#11993) 2023-06-15 11:15:52 +05:30
Mayur Singal
7fa963eec3
Fix #1076: Add mongodb support (#11943) 2023-06-15 11:14:22 +05:30
Sriharsha Chintalapani
3c62ddc561
Fix security configuration docs (#11992) 2023-06-14 19:51:57 -07:00
YAMADA Yutaka
63e8aa57ce
Fixes #11955: Modify URL to register with Airflow Lineage Backend (#11956)
* fix: include host_port in url

* fix: run make py_format
2023-06-14 14:06:01 +02:00
Onkar Ravgan
d409c339e4
Fix 10689: Added Powerbi Datamodels (#11923) 2023-06-14 17:00:08 +05:30
Akash Jain
78eaa06fab
fix(docker): Syntax issues (#11982) 2023-06-14 10:25:37 +05:30
Suresh Srinivas
7d9b27b996
Fixes #11976 Add support for suggest and request description and tags for all the entities (#11977) 2023-06-13 12:45:07 -07:00
Akash Jain
c96e239b36
fix(docs): Update Airflow Docker environment variables (#11974) 2023-06-13 10:38:32 -07:00