58 Commits

Author SHA1 Message Date
Onkar Ravgan
d4f9f0f875
updated oracle permissions (#11024) 2023-04-13 14:22:58 +05:30
Ayush Shah
288358952b
Fixes 8264: add docs around adding taxonomy project id (#11018)
* add docs around taxonomy project id

* Add taxonomyProjectID to bigquery.yaml
2023-04-12 16:00:05 +05:30
NiharDoshi99
325480fb2e
added migration scripts for druid and postgres docs update (#10920)
* added migration scripts for druid and postgres docs update

* postgres docs update
2023-04-04 22:45:19 +05:30
Mayur Singal
69729ac8eb
Fix #10320: Add Athena Usage & Lineage (#10879) 2023-04-03 11:21:55 +00:00
Sachin Chaurasiya
0a92a897a1
chore(ui): add support for service documentation (part-1) (#10668)
* chore(ui): add support for service documentation md file

* sync local

* chore: add method for fetching markdown file

* chore(ui): add support for service documentation

* chore: move fields to connections

* chore: update logic to fetch requirements

* chore: right panel component for service

* fix: key prop is not present in the skeleton component

* chore: only fetch md files when required fields are present

* chore: use hook for fetching airflow status

* chore: refactor add service component

* chore: remove id prefix and id separator prop from form builder

* fix: fieldName issue on right panel

* fix: active Field name issue

* fix:unit test

* test: add unit test

* chore: handle edit service form

* chore: add fallback logic

* fix: cy test

* chore: update service doc md files/folder structure,

* chore: push image example

* Athena docs

* Add glue docs

* Add hive related changes

* chore: take last field for fetching field doc

* add datalake

* Added connection information for oracle and redshift (english + french)

* fix: fallback logic

* Bigquery & Snowflake Requirements

* mysql and amundsen requirements (#10752)

* Revert removal of descriptions

* Add Doc For Mssql and Postgres

* Added powerbi conn md files

* Align requirements files

* Add Kafka and Redpanda

* refined powerbi docs

* Add Tableu requirements, move Athena and Glue fields, change footer some connectors

* Add missing connectors fields descriptions default

* re: datalake

* Add Tableau field descriptions

* fix: markdown styling

* chore: improve button styling

* chore: rename right panel to service right panel and move it to common

* fix: doc for select and list field , cy test.

* fix: unit test

* fix: test connection service type issue

* Added powerbi docs link in req

* Add info on hive

* Remove unused markdowns

* Add req for datalake

* add hive requirements header

* Snowflake & Biguqery

* Update Mssql and Postgres

* mysql and amundsen requirements updated

* Update Mssql and Postgres

* added username

* chore: fix cy expression issue

* chore: reset active field state on step change.

* fix: affix target container issue

* fix: unit test

* fix: cypress for postgres and glue

---------

Co-authored-by: Milan Bariya <52292922+MilanBariya@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: NiharDoshi99 <51595473+NiharDoshi99@users.noreply.github.com>
Co-authored-by: Milan Bariya <milanbariya12@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
2023-03-29 14:18:17 +05:30
Onkar Ravgan
5d6e18dc28
Fix 10642: Mark delete entities and tags toggle (#10695)
* Added mark delete logic

* Final test and optimization

* After merge fixes

* Added include tags for dash pipelines dbt

* added docs and fixed test

* Fixed py tests

* Added UI changes for following newly added fields:
- markDeletedDashboards
- markDeletedMlModels
- markDeletedPipelines
- markDeletedTopics
- includeTags

* Fixed failing unit tests

* updated json files of localization for other languages

* Improved localization changes

* added localization changes for other languages

* Updated mark deleted desc

* updated the ingestion fields descriptions in the ingestion form for UI

* automated localization changes for other languages

* updated descriptions for includeTags field for dbtPipeline and databaseServiceMetadataPipeline json

* fixed issue where includeTags field was being sent in the dbtConfigSource

* Added flow to input taxonomy while adding BigQuery service.

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2023-03-29 12:41:44 +05:30
Milan Bariya
3d1bbb1037
Add a configurationto skip Snowflake transient and tmp tables (#10665)
* Add a configurationto skip  Snowflake transient and tmp tables

* Fix Python checkstyle

* add separate query for transient tables

* Move skipTempTables into SnowflakeConnection

* Fix merge conflict

* change skip word to include

* Add title in json file

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-28 06:29:59 +02:00
NiharDoshi99
bfa0905093
update druid docs (#10735) 2023-03-23 22:16:14 +05:30
Mayur Singal
9f99296ea0
Bigquery Docs Fix (#10692)
* Bigquery Docs Fix

* typo fix

* keycloak sso

* GCP Create Custom Role
2023-03-21 16:55:57 +01:00
Ayush Shah
133bdfbf64
Add Okta Doc fix, Athena lineage fix, quicksight permissions (#10683) 2023-03-21 12:44:58 +05:30
Milan Bariya
fadefff3e1
Fix databricks timeout issue (#10613)
* Fix databricks timeout issue

* Change based on comments

* Change timeOut word to connectionTimeout
2023-03-16 16:13:49 +01:00
NiharDoshi99
1ff76f5e65
pii tagging using spacy (#10256)
* WIP: pii tagging using spacy

* added test cases and changes as per comment

* fix python checkstyle

* fix python checkstyle

* added score, test_cases and docs update

* solved merge conflict

* fix python checkstyle

* remove pii tagging using regex

* fix python test

* lib changes and added some test case

* changed as per comment

* fix: python test

* fix: changes to get source_config

* fix: changes as per comment
2023-03-03 18:33:18 +05:30
Teddy
754074f1be
Fixes #7758 - Added Column value and Integer Range Partitionning (#10350)
* feat(profiler): renamed  module to

* feat(profiler): added dbt-artifacts-parser to test setup.py

* feat(profiler): refactor workflow and interface

* feat(profiler): linting

* feat(profiler): removed old profiler modules

* feat(profiler): added support for value and integer range partition

* feat(profiler): fixed linting

* feat(profiler): added partitionning support for datalake profiler

* feat(profiler): removed `ProfilerInterfaceArgs` class

* feat(profiler): address comments

* feat(profiler): Added `OTHER` as an `IntervalType` for UI type generation
2023-03-01 08:20:38 +01:00
kcd83
c0f06191b0
Tips on how to format and private key for the UI and in YAML (#10370)
Co-authored-by: kcd <1108143-kcd@users.noreply.gitlab.com>
2023-02-28 12:18:21 -08:00
Mayur Singal
9e6a91d349
Glue Req Docs (#10049) 2023-02-01 14:41:11 +01:00
Ayush Shah
4e0598db44
Fix Docs (#10035) 2023-01-31 21:26:26 +05:30
Ayush Shah
747fcf569b
Add docs - quicksight, lineage... (#10023) 2023-01-31 15:17:40 +00:00
Teddy
0b3996d2fa
Remove profiler support for dynamodb (#10014)
* fix(DynamoDB): remove profiler support and cleaned up documentation

* fix(DynamoDB): remove supportProfiler filed in tests
2023-01-31 08:29:35 +01:00
Mayur Singal
399adf788d
Oracle & Superset Req Docs (#10015) 2023-01-31 07:07:25 +01:00
NiharDoshi99
c81446e2be
updated domo docs (#10005)
* updated domo docs

* changed as per comment
2023-01-30 20:59:35 +05:30
Pere Miquel Brull
c7c3d62bae
Update profiler permissions (#10004) 2023-01-30 14:20:19 +01:00
Pere Miquel Brull
0b8523f418
Docs - Add select on views (#10000) 2023-01-30 11:46:58 +00:00
Nahuel
da1f911581
Doc: Add missing source config types in connectors config examples (#9955) 2023-01-27 15:30:48 +01:00
Pere Miquel Brull
9160770a47
Docs cleanup for dbt YAML & Looker config (#9937)
* Update looker config

* cleanup dbt
2023-01-27 09:06:30 +01:00
Pere Miquel Brull
4650a453e1
Prepare Vertica Lineage and Usage (#9906)
* Prepare Vertica Lineage and Usage

* Simplify db usage

* Linting

* Revert postgres changes

* Revert postgres changes

* Add vertica flags
2023-01-26 14:34:33 +01:00
NiharDoshi99
a817b18104
updated postgres docs (#9920) 2023-01-25 21:04:25 +05:30
Milan Bariya
49d48e0546
Update: Trino, Presto Logic and Doc (#9859)
* Update: Trino, Presto Logic and Doc

* Update: Trino, Presto Logic and Doc

* Update permissions in Doc

* Update openmetadata-docs/content/connectors/database/trino/index.md

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-01-25 19:20:06 +05:30
Mayur Singal
6b4e500fa6
Update Redshift Requirement Docs (#9902) 2023-01-25 14:08:48 +05:30
Mayur Singal
b3f9e3ba97
Update snowflake Requirement Docs (#9900) 2023-01-25 11:05:14 +05:30
Pere Miquel Brull
16a1b2c8be
Vertica comments, dbs, profiler and docs (#9845)
* Vertica comments, dbs, profiler and docs

* Revert metabase changes

* Format

* Fix median
2023-01-20 20:36:03 +01:00
Pere Miquel Brull
b0058bdb65
Fix #9707 - Install instantclient and enable thick mode (#9761)
* Install instantclient

* Prep instant client

* Update docs

* Amend comments
2023-01-19 08:08:56 +01:00
Ayush Shah
03c40d9566
Add Doc for Data Catalog api permission (#9676) 2023-01-10 19:48:12 +01:00
Suresh Srinivas
758c976cba
Fixes #9259 Change Tags APIs to conform with rest of the APIs (#9260) 2022-12-26 12:32:17 -08:00
Onkar Ravgan
14fa96958f
Added dbt workflow docs (#9493)
* Added dbt workflow docs

* added dbt small case

* Fixed review comments
2022-12-22 13:11:18 +00:00
Ayush Shah
b3d7719289
Add Azure Datalake to the list (#9487)
* Add Azure Datalake to the list

* Put Yaml configs under the relevant sections
2022-12-22 11:15:57 +01:00
Ayush Shah
ba94f78aaa
Add docs for data quality (#9457) 2022-12-21 13:41:37 +00:00
NiharDoshi99
9a3d599f30
Azure datalake minor changes (#9407) 2022-12-21 10:28:41 +05:30
Milan Bariya
a3495c67f0
Databricks improve code and doc update (#9394)
* Databricks improve code and doc update

* Change base on comments

* Fix pytest fail
2022-12-19 18:53:10 +05:30
NiharDoshi99
4619ade3b5
Fix: updated docs for datalake-azure (#9358)
* Fix: updated docs for datalake-azure

* Fix: as per comment
2022-12-17 16:22:03 +05:30
Teddy
601b9cd38a
Added SSL Connection steps for Clickhouse connection (#9210) 2022-12-08 13:40:11 +01:00
Pere Miquel Brull
90ae3a6984
Postgres version requirements (#9070) 2022-11-30 14:59:48 +05:30
Teddy
69babdc309
Added data insights docs + update profiler CLI to include partition (#9010) 2022-11-25 14:19:26 +01:00
Mayur Singal
859cfec5d0
Azure docs fix (#8928)
* Azure Docker Docs

* Add Image
2022-11-21 15:44:41 +01:00
Mayur Singal
625b0551b1
Added File Types in Databrick Docs (#8865) 2022-11-19 21:07:48 -08:00
NiharDoshi99
185872ca08
Fix: Updated docs for domo bigquery (#8843) 2022-11-18 19:15:42 +05:30
Pere Miquel Brull
fe16dea584
Fix #8794 - Separate DL requirements and lazy imports (#8806) 2022-11-17 09:11:54 +00:00
Meet Parita Munesh Shah
9c360fb61c
added docs for DOMO (#8543)
* added docs for DOMO

* Changed As Per Comment

* Change as per comment

* Changes as per comment

Co-authored-by: Meet <meet.s@deuexsolutions.com>
2022-11-09 16:44:02 +05:30
Milan Bariya
158bd4b9cd
Add: databricks troubleshoot page (#8523) 2022-11-03 14:27:50 +01:00
clueless-bot
68c3e0b8fe
FIX: Add postgres usage and lineage details (#8456) 2022-11-01 00:18:05 +05:30
Pere Miquel Brull
430f9d9b54
AzureSQL troubleshooting (#8427) 2022-10-29 09:35:33 +02:00