10673 Commits

Author SHA1 Message Date
Ashish Gupta
783f8e3c33
fix search index value for search and data product (#13441) 2023-10-05 15:25:03 +05:30
Mayur Singal
0090286924
Fix Bigquery Test connection for multiproject (#13380)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-10-05 14:50:42 +05:30
Mayur Singal
f879656f0a
Fix #12047: Clean commonregex package from setup (#13439) 2023-10-05 13:41:31 +05:30
Teddy
c4a3de6a85
fix: handle tableConfig for profiler CLI (#13437)
* fix: handle tableConfig for profiler CLI

* fix: empty commit for CI
2023-10-05 10:02:57 +02:00
Teddy
ddae3d8143
Refactor Data Insight aggregators Classes (#13433)
* fix: removed legacy OS and ES aggregator classes

* fix: centralized aggregator business logic

* fix: implemented client specific aggregator

* fix: updated client instantiation to use client specific aggregator

* fix: clean up json schema

* fix: updated DI index names

* fix: added searchIndex + storedProcedure

* fix: ran linting

* fix: updated python test to include new entity types
2023-10-05 09:31:27 +02:00
Vlad
c505b638d0
Update references to Airflow instead of Airbyte (#13438) 2023-10-04 13:44:40 -07:00
Ashish Gupta
6bbb274e11
supported drag and drop to glossary level for glossary terms (#13434) 2023-10-04 22:01:54 +05:30
Shailesh Parmar
b478039207
feat(ui): Add Badge with color or icon for Tags,GlossaryTerm, Domain and DataProducts #9972 (#13349)
* feat(ui): Add Badge with color or icon for Tags,GlossaryTerm, Domain and DataProducts #9972

* added api call to patch style in glossary

* updated and refactor AsyncSelectList and tagsContainerV2

* updated tags in all the pages

* added style object in addGlossaryTerm form

* added style support in domain and tags

* fixed failing unit test

* added cypress for the style feature

* added unit test for new component

* translation-sync

* addressing comment

* fixed edit tags issue

* added style feature for domain

* fixed sonarLint

* fixed merge issue
2023-10-04 21:50:39 +05:30
Sachin Chaurasiya
be278b67ac
chore(ui): add route and sidebar prop support in app (#13430)
* chore(ui): add children support in app

* fix: assets path

* chore: add support for route and sidebar elements prop

* chore: remove children prop from app
2023-10-04 19:05:48 +05:30
Nguyen Huu Loc
ef1974edd6
Support LookML multi repos (#13140)
* Draft: Support LookML multi repos

* [Looker] manually create Dashboard datamodel

* [Looker] Support remote import & lineage for looker view

* Rollback parser.py

* refactor code

* Update code

* Remove logs & add comments

* Remove Middle & Nothing

* - Fix yield datamodel error
- Remove logs

* Support clone repo from Bitbucket

* Fix typo

* Optimize imports

* Fix pylint

---------

Co-authored-by: Loc Nguyen <loc.nguyenhuu@xendit.co>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-10-04 15:16:21 +02:00
Ayush Shah
97f4f8fbf3
Fixes 12922: Trino NaN issue + TrinoUserError (#13244)
* Fix Trino NaN issue + TrinoUserError
2023-10-04 18:39:39 +05:30
RyoAriyama
b2ee1a54ef
fix return type of docstring powerbi (#13422) 2023-10-04 15:00:06 +02:00
Anatoliy Shulika
b788061157
fixes #12771: Added Greenplum Ingestion Connector (#13128)
* ISSUE-12771: Added Greenplum Ingestion Connector

* fixed python code formating

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-10-04 14:53:53 +02:00
mitchellmann
0ba02a5977
Fixes 13249: Added ingestion support - Presto tbl/col comments (#13250)
* Added ingestion support - Presto tbl/col comments

* now supports scenario of NONE schema

* PY style fixes

---------

Co-authored-by: Mitchell Mann <mitchell.mann@tideworks.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-10-04 14:41:12 +02:00
Mayur Singal
3b640b43b7
Fix column lineage nonetype error (#13432) 2023-10-04 17:48:56 +05:30
Ashish Gupta
2e49b72ccf
supported removal of subscription in teams (#13404)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-10-04 17:05:28 +05:30
Ashish Gupta
eb89d789bf
fix(ui): supported announcement in pending entities (#13414)
* supported announcement in pending entities

* added announcement support in database, database schema and entity services
2023-10-04 17:05:14 +05:30
Akash Jain
5a3d759b48
fix(CI): Cypress CI timeout fixes (#13431)
* fix(CI): Increase Server Start timeout to 60 minutes

* fix(CI): timeout on cypress step instead of branch
2023-10-04 15:14:43 +05:30
Onkar Ravgan
13f2d3e25e
Added entity_ref instead of fqn in unused data assets aggregrate API (#13421) 2023-10-04 15:07:09 +05:30
Pere Miquel Brull
0282574bdd
Create ometa client once and pass it around & improve pycln config (#13310)
* Create ometa client once and pass it around & improve pycln config

* Fix

* Fix

* Fix tests

* Fix maven ci

* Fix tests

* Fix tests

* Fix tests

* Format

* Fix DI
2023-10-04 09:14:03 +02:00
Sriharsha Chintalapani
1a90c5c1b3
Refactor Repositories: Remove passing around CollectionDAO (#13427)
* Refactor repository registration

* Refactor repository registration

* Refactor repository registration

* Refactor repository registration
2023-10-03 23:41:06 -07:00
Pere Miquel Brull
31b827585b
Allow ometa to create services without storing the connection (#13400)
* Allow ometa to create services without storing the connection

* Allow ometa to create services without storing the connection

* Fix backend tests with null connection
2023-10-04 07:48:49 +02:00
dependabot[bot]
77912f9751
chore(deps): bump quill-mention (#13377)
Bumps [quill-mention](https://github.com/quill-mention/quill-mention) from 3.1.0 to 4.0.0.
- [Commits](https://github.com/quill-mention/quill-mention/commits)

---
updated-dependencies:
- dependency-name: quill-mention
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-10-04 11:14:51 +05:30
Mayur Singal
4f4d1c725c
Fix failing E2Es (#13419) 2023-10-04 10:56:34 +05:30
Sriharsha Chintalapani
eb84c524aa
Fix repository registration (#13425)
* Fix failing repository registration in external

* Fix failing repository registration in external

* Scan classPath to find Repository annotation

* Remove additioanl config, register any classes with Repository or Collection annotation

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-10-03 21:03:56 -07:00
Sriharsha Chintalapani
8578aee04f
Search refactor (#13397)
* Refactor Search

* Refactor Search

* Fix propgation bugs

* Fix propgation bugs

* Fix glossary term search

* Fix glossary term search

* Only issue search requests if the client is configured properly

* Only issue search requests if the client is configured properly

* Fix glossary index

* add documentation for sharded tables (#13361)

* Cost analysis agg (#13408)

* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes

* feat: __init__.py files creation

* feat: updated workflow import classes in code and doc

* feat: moved kpi runner from runner to processor folder

* fix: skip failure on list entities

* feat: deleted unused files

* feat: updated status reporter

* feat: ran linting

* feat: fix test error with typing and fqn

* feat: updated test dependencies

* feat: ran linting

* feat: move execution order up

* feat: updated cost analysis report to align with new workflow

* feat: fix entity already exists for pipeline entity status

* feat: ran python linting

* feat: move skip_on_failure to method

* feat: added unusedReport to DI

* feat: added aggregated unused report

* feat: ran linting

* feat: reverted compose file changes

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>

* Add Java 17 support (#12895)

* Add Java 17 support

* Change Test HTTP client provider

* Create Tests HTTP Client once

* Create Tests HTTP Client once

* fix(CI): Update CI to use jdk 17 and dockerfiles as well

---------

Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>

* Refactor Search

* Refactor Search

* Fix propgation bugs

* Fix propgation bugs

* Fix glossary term search

* Fix glossary term search

* Only issue search requests if the client is configured properly

* Only issue search requests if the client is configured properly

* Fix glossary index

* Merge main

* fix style

* deleted field propagation

* Fix style

* close the test client

* Changing to jersey connector

* Fix Authentication Exception headers

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2023-10-03 14:56:53 -07:00
Shailesh Parmar
efa0802995
ui: UI to use Tier TAG displayName if provided #12899 (#13105)
* ui: UI to use Tier TAG displayName if provided #12899

* updated tier label with displayName
2023-10-04 01:26:14 +05:30
karanh37
b5aff6bbc3
fix: encode search query text (#13420) 2023-10-03 19:52:19 +05:30
Sachin Chaurasiya
0ecc3fc136
chore(ui): change baseUrl in ts config (#13403)
* chore(ui): change baseUrl in ts config

* udpate references

* fix rest/* path

* fix tests paths to relative

* chore(ui): update reference

* fix unit tests

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-10-03 16:01:13 +05:30
Sriharsha Chintalapani
03ca972887
Add Java 17 support (#12895)
* Add Java 17 support

* Change Test HTTP client provider

* Create Tests HTTP Client once

* Create Tests HTTP Client once

* fix(CI): Update CI to use jdk 17 and dockerfiles as well

---------

Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2023-10-03 01:35:50 -07:00
Teddy
9ef3ff7a58
Cost analysis agg (#13408)
* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes

* feat: __init__.py files creation

* feat: updated workflow import classes in code and doc

* feat: moved kpi runner from runner to processor folder

* fix: skip failure on list entities

* feat: deleted unused files

* feat: updated status reporter

* feat: ran linting

* feat: fix test error with typing and fqn

* feat: updated test dependencies

* feat: ran linting

* feat: move execution order up

* feat: updated cost analysis report to align with new workflow

* feat: fix entity already exists for pipeline entity status

* feat: ran python linting

* feat: move skip_on_failure to method

* feat: added unusedReport to DI

* feat: added aggregated unused report

* feat: ran linting

* feat: reverted compose file changes

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-10-03 09:27:18 +02:00
Ayush Shah
744e5279bc
add documentation for sharded tables (#13361) 2023-10-03 12:49:05 +05:30
Onkar Ravgan
bc491be5ad
Fixed dbt optional files for local config (#13242) 2023-10-03 12:44:06 +05:30
Ayush Shah
462b2f9445
Fix Latest Pylint 3.0.0 issues (#13413)
* Fix Latest Pylint issues

* add compatible bound to pylint version
2023-10-03 07:43:09 +02:00
Pere Miquel Brull
b5596a4640
Batch PII tagging (#13385)
* Batch PII tagging

* Batch PII tagging

* Fix tests

* Fix tests
2023-10-02 14:44:41 +02:00
Pere Miquel Brull
5acf837e23
Fix #12870 - Troubleshoot Debian/CentOS Airflow apis installation (#13292) 2023-10-02 14:44:28 +02:00
Ashish Gupta
16a4033645
fix(ui): revamp teams page added supported subscription webhooks (#13296)
* revamp teams page added supported subscription webhook

* minor changes

* minor changes

* changes teams header page layout and subscription

* minor changes

* fix cypress and addressed comments

* fix cypress for teams hierarchy (#13352)

* fix sonar errors and users not showing in teams having space

* code smell and bugs fixes

* fix teams page cypress

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-10-02 18:12:23 +05:30
Ashish Gupta
e879d512d3
fix(ui): initial load tab details in container page (#13399)
* fix intial load tab details in container page

* fix update displayName tests

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-10-02 16:26:01 +05:30
Pere Miquel Brull
d915254fac
Prepare Storage Connector for ADLS & Docs (#13376)
* Prepare Storage Connector for ADLS & Docs

* Format

* Fix test
2023-10-02 12:15:09 +02:00
Teddy
6ca71ae323
Issue 12679 - Handle Entity Object Instantiation Error + Refactor Workflow (#13384)
* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes

* feat: __init__.py files creation

* feat: updated workflow import classes in code and doc

* feat: moved kpi runner from runner to processor folder

* fix: skip failure on list entities

* feat: deleted unused files

* feat: updated status reporter

* feat: ran linting

* feat: fix test error with typing and fqn

* feat: updated test dependencies

* feat: ran linting

* feat: move execution order up

* feat: updated cost analysis report to align with new workflow

* feat: fix entity already exists for pipeline entity status

* feat: ran python linting

* feat: move skip_on_failure to method

* feat: ran linting

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-10-02 12:05:30 +02:00
karanh37
41f56d62af
fix: add more button in feeds widget (#13341)
* fix: add more button in feeds widget

* fix: add feed filter

* fix: add localisation

* fix: add filter dropdown in feeds widget

* fix: missing localisation

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-10-02 15:14:39 +05:30
Chirag Madlani
241c838c64
fix(test): cypress skipped tests (#13360)
* fix(test): cypress skipped tests

* limit cypress tests to changed one

* fix service search

* enable cypress tests

* enable tier and owner spec

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-10-02 12:06:10 +05:30
Sriharsha Chintalapani
11f1df437b
Fix SettingsCache initialization (#13395)
* Fix SettingsCache initialization

* Fix SettingsCache initialization

* Fix SettingsCache initialization
2023-10-01 15:47:25 -07:00
Teddy
66d3fbb75d
feat: added german translation for OM (#13386) 2023-10-01 22:33:12 +05:30
Suresh Srinivas
2e2f9a1edb
Separate initialization of Repository and Resources (#13389)
* Separate initialization of Repository and Resources

* Change that speeds up the test. Still WIP

* Rebasing the branch with main

* Revert back previous changes and use Repositories from registry for migration

* Fixing test failures

* Code cleanup

* Fix test failures
2023-09-30 22:13:16 -07:00
Aniket Katkar
79558ab1a6
added quick links in how to guide homepage (#13390) 2023-09-29 21:10:55 -07:00
Sriharsha Chintalapani
762824665a
Fix #12655: Customizable Landing Page - Add Personas, DocStore to support KnowledgePanels and Pages (#13229)
* Fix #12655: Customizable Landing Page - Add Personas

* Add Knowledge Panels

* Add Doc Store to store KnowledgePanels and Pages

* Add Doc Store to store KnowledgePanels and Pages schemas

* Fix tests

* Add Multiple Persona support for a user

* add more elaborate tests for docstore

* Fix Page and KnowledgePanel required fields

* Fix Page and KnowledgePanel required fields

* Fix Page and KnowledgePanel required fields

* Fix build
2023-09-29 19:13:56 -07:00
Sriharsha Chintalapani
3a7f7486d0
Add indexes to name column for efficient ORDER BY name queries (#13383) 2023-09-29 14:19:49 +02:00
Onkar Ravgan
855790924e
Cost Analysis - Data Insights Reports and Aggregation (#13379)
* Added reports and indexes

* Clean code

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-09-29 11:50:43 +02:00
Teddy
162b181b4f
fix: pass metadata object to interface (#13339) 2023-09-29 08:02:02 +02:00