1453 Commits

Author SHA1 Message Date
Mayur Singal
e9992a52a8
Fix #1604: Add Spline Pipeline Connector (#11562)
* Fix #1604: Add Spline Connector

* Add tests & grammer validation

* Spline UI Changes & Docs

* fix pipeline workflow doc

* chore: use common field for dbService name

* chore: use const for beta services

* chore: add service icon

* Update ingestion/src/metadata/ingestion/source/pipeline/spline/metadata.py

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-05-19 14:46:32 +05:30
Pere Miquel Brull
b60f198674
Fix GitHub reader headers and add docs (#11675)
* Fix header and add docs

* Format
2023-05-19 09:36:30 +02:00
Pere Miquel Brull
667706d09b
[Docs] - Update Airflow SSL docs (#11621)
* Update Airflow SSL docs

* Update openmetadata-docs/content/v1.1.0-snapshot/deployment/security/enable-ssl/airflow-ssl.md

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-18 14:51:40 +05:30
Mayur Singal
e4997c3749
Fix #11571: Support custom database name for glue (#11631) 2023-05-18 14:16:56 +05:30
Onkar Ravgan
e6a7ffc57c
Fixes: 11540 Added api_version and domain fields to Salesforce (#11618)
* Added sales new fields

* fixed image after merge

* Fix Failing Test

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-05-18 10:53:44 +05:30
Pere Miquel Brull
2008eda764
[Docs] - Rename to v1.1.0-snapshot and fix sync CI (#11644)
* Rename v1.1.0 to v1.1.0-snapshot

* Delete old docs
2023-05-17 15:55:31 +02:00
Sriharsha Chintalapani
6ca1ec6fbe
Delete old docs (#11627)
* Delete old docs and rename the openmetadata-docs-v1 to openmetadata-docs

* Delete old docs and rename the openmetadata-docs-v1 to openmetadata-docs

* Delete old docs and rename the openmetadata-docs-v1 to openmetadata-docs
2023-05-17 07:04:56 +02:00
Mads Emil
b0764aa7dd
Change presistent to persistent and delpoy to deploy (#11541) 2023-05-10 18:14:07 -07:00
Ayush Shah
70b09d1691
Fix impala (#11535)
* Add impala docs

* Add impala

* remove externalbrowser

* Add Config doc

* Add config options

* Add config options
2023-05-10 21:46:49 +05:30
Mayur Singal
e20f75fa54
Fix #11468: Fix salesforce scheme issue (#11474) 2023-05-08 14:02:17 +05:30
Sriharsha Chintalapani
5978e9a48b
Revert "Fix docs (#11443)" (#11446)
This reverts commit 9a0ab3ba583bbc670610e899ba221ac3d941027f.
2023-05-04 12:37:18 -07:00
Ayush Shah
9a0ab3ba58
Fix docs (#11443)
* Fix FiveTran Docs

* Fix Custom Logo doc

* Fix Glue Pipeline

* Fix Docs

* Fix Dagster

* Fix Kafka

* Fix Connector References

* Fix Readme

* Fix link

* Add v1.0.0
2023-05-04 23:26:50 +05:30
Mayur Singal
fd5f63fb58
Fix MSSQL connection with pyodbc scheme (#11304)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-04-27 07:25:10 +02:00
Ayush Shah
8eb2a24721
Add Dashboard docs (#11278) 2023-04-26 17:41:42 +05:30
Mayur Singal
d0781dcc07
Remove partition fields from Bigquery (#11195)
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-04-23 06:38:17 +02:00
Pere Miquel Brull
67e1fa2810
[Docs] - Looker & S3 (#11184)
* Looker docs

* S3 docs

* Fix roadmap
2023-04-22 18:22:30 +02:00
Ayush Shah
cdef9e54ee
Add dagster docs (#11198) 2023-04-22 10:04:20 +05:30
Ayush Shah
fa39f2732a
Mssql docs (#11196) 2023-04-21 20:20:13 +05:30
Ayush Shah
c14ef44d5c
Add Oracle Docs (#11193) 2023-04-21 20:19:53 +05:30
Mayur Singal
e494cdfb65
Ingestion Docs: Superset, Snowflake, Bigquery, Azure SQL (#11180)
* Snowflake, Bigquery & Azuresql docs with superset migrations

* Superset Docs

* Auto PII is not a part of ingestion

* bigquery docs

* Azure SQL Docs

* Snowflake Docs

* Superset Docs

* Url to URL

* replace hr with ---
2023-04-21 18:23:45 +05:30
Ayush Shah
d6aa49520a
Clickhouse Docs (#11182) 2023-04-21 16:05:21 +05:30
Ayush Shah
9a0fb262b7
Add Quicksight docs (#11179) 2023-04-21 15:57:31 +05:30
Keith Sirmons
97b58c65f5
Impalaconnection (#11151)
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.

* added the ordinalPosition data point into the Column constructor.

* renamed variable to better describe its usage.

* updated profile errors.
Hive connections now comment columns by default.

* removed print statements

* Cleaned up code by pulling check into its own function

* Updated median function to return null when it is being used for first and third quartiles.

* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.

* added the ordinalPosition data point into the Column constructor.

* renamed variable to better describe its usage.

* updated profile errors.
Hive connections now comment columns by default.

* removed print statements

* Cleaned up code by pulling check into its own function

* Updated median function to return null when it is being used for first and third quartiles.

* removed print statements and ran make py_format

* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine

* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala

* added cast to decimal for column when running average in mean.py

* fixed lint error

* fixed ui ordering of precision and scale.
Precision should be ordred in front of scale since the precision is set first in decimal data types

* first pass for impala connector

* updated default auth_mechanism to be one of the enum values.

* updated UI documentation to match fields for the impalaconneciton.

refined impalaConnection to bring use_ssl to a boolean instead or relying on an extra connection option being manually added.

Removed reference to hive for type mapping

added impala to the pip setup

* py_format updates

* removed print statement

* Lints and fixes

* Updated database documentation to follow new style

* Flag as BETA

* Remove tests

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-04-21 09:57:13 +02:00
Nahuel
8184516f80
Fix: Align includeOwner option with the rest of options to include entities (#11160) 2023-04-20 16:01:54 +02:00
NiharDoshi99
1862ba2ba4
Changing behaviour for owners same as description for dashboards (#11118)
* changing behaviour for owners same as description

* fix typo
2023-04-19 12:31:56 +05:30
Pere Miquel Brull
3109c07e4c
Update roadmap release date (#11099) 2023-04-18 07:46:49 +02:00
dhruvinmaniar123
92d8197f0b
K8 step update (#11051)
* Update eks.md

* Update gke.md

* Update on-prem.md

* Update eks.md

* Update gke.md

* Update on-prem.md

* Update gke.md

* Update gke.md
2023-04-17 20:43:36 +05:30
Nahuel
ea70580aff
Fix#10584: Tableau E2E and docs (#11054) 2023-04-14 19:28:31 +05:30
Nahuel
c596ad2006
Doc: Update Kafka troubleshooting section (#11067) 2023-04-14 13:35:58 +02:00
Onkar Ravgan
bc6ce22a2b
Added oneof selection for tableau auth types (#11049)
* Added tableau oneof fields

* Fixed pytests

* fixed field in test

* Handle tableau auth converter

* Fixed java tests and imports

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-04-14 13:49:36 +05:30
Teddy
77b94f9ebb
fix: rename tests endpoint to dataQuality/<specificity> (#10970) 2023-04-14 00:14:49 -07:00
Sriharsha Chintalapani
c74642b71e
Update latest-release.md (#11050)
* Update latest-release.md

* Update index.md
2023-04-13 08:08:12 -07:00
Milan Bariya
dda5963e34
Data insights with metadata command-line (#11045)
* Data insights with metadata command-line

* Python Checkstyle

* Python Checkstyle
2023-04-13 13:42:07 +00:00
Onkar Ravgan
d4f9f0f875
updated oracle permissions (#11024) 2023-04-13 14:22:58 +05:30
Ayush Shah
288358952b
Fixes 8264: add docs around adding taxonomy project id (#11018)
* add docs around taxonomy project id

* Add taxonomyProjectID to bigquery.yaml
2023-04-12 16:00:05 +05:30
Pere Miquel Brull
c6f31ed406
Docs - Ingestion Framework Debug settings (#11009) 2023-04-12 11:23:31 +02:00
Milan Bariya
ff3e19498a
Doc update for Data-insight (#10985) 2023-04-11 12:18:00 +02:00
Sriharsha Chintalapani
7c6b9b3c56
Update python sdk client connection details (#10963)
* Update python sdk client connection details

* Update index.md
2023-04-06 20:44:36 -07:00
NiharDoshi99
325480fb2e
added migration scripts for druid and postgres docs update (#10920)
* added migration scripts for druid and postgres docs update

* postgres docs update
2023-04-04 22:45:19 +05:30
Teddy
9b4e9132ae
fixed #9656 - Add support for date type to column values to be between (#10890)
* fix: renamed  to  submodule

* fix: linting

* fix: columnValuesToBeBetween test for date column type
2023-04-04 17:16:44 +02:00
Sriharsha Chintalapani
e6ac7af49d
Fix roadmap dates and features (#10922) 2023-04-04 16:48:11 +02:00
Onkar Ravgan
608bf51b6a
Added non admin apis support for PowerBI (#10910)
* Added non admin apis support

* addressed review comments
2023-04-04 18:25:48 +05:30
Suresh Srinivas
c8b640674b
10041 part2 - Refactor and cleanup APIs (#10900)
* Use @Tag annotation to group APIs in the swagger documentation.

* Hide internal APIs

* Change API path events/subscription to events/subscriptions

* Change API path from automations/workflow to automations/workflows

* Change API path v1/testCase to v1/testCases

* Change API path v1/testDefinition to v1/testDefinitions

* Change API path v1/testSuite to v1/testSuites

* Rename Kpi and kpi in the documentation to KPI

* Change API path v1/testConnectionDefinition to v1/testConnectionDefinitions

* Update API section in the API documentation

* Fix test failures

* Correctly capitalize Test Cases and Test Suites in API docs
2023-04-03 13:03:48 -07:00
Mayur Singal
69729ac8eb
Fix #10320: Add Athena Usage & Lineage (#10879) 2023-04-03 11:21:55 +00:00
Sachin Chaurasiya
ae5e7a3d98
doc: update rule doc to include conditions (#10845) 2023-03-31 18:36:09 +05:30
Chirag Madlani
f7f5040008
update node version in prerequisites (#10844) 2023-03-30 13:07:49 +00:00
Sachin Chaurasiya
0a92a897a1
chore(ui): add support for service documentation (part-1) (#10668)
* chore(ui): add support for service documentation md file

* sync local

* chore: add method for fetching markdown file

* chore(ui): add support for service documentation

* chore: move fields to connections

* chore: update logic to fetch requirements

* chore: right panel component for service

* fix: key prop is not present in the skeleton component

* chore: only fetch md files when required fields are present

* chore: use hook for fetching airflow status

* chore: refactor add service component

* chore: remove id prefix and id separator prop from form builder

* fix: fieldName issue on right panel

* fix: active Field name issue

* fix:unit test

* test: add unit test

* chore: handle edit service form

* chore: add fallback logic

* fix: cy test

* chore: update service doc md files/folder structure,

* chore: push image example

* Athena docs

* Add glue docs

* Add hive related changes

* chore: take last field for fetching field doc

* add datalake

* Added connection information for oracle and redshift (english + french)

* fix: fallback logic

* Bigquery & Snowflake Requirements

* mysql and amundsen requirements (#10752)

* Revert removal of descriptions

* Add Doc For Mssql and Postgres

* Added powerbi conn md files

* Align requirements files

* Add Kafka and Redpanda

* refined powerbi docs

* Add Tableu requirements, move Athena and Glue fields, change footer some connectors

* Add missing connectors fields descriptions default

* re: datalake

* Add Tableau field descriptions

* fix: markdown styling

* chore: improve button styling

* chore: rename right panel to service right panel and move it to common

* fix: doc for select and list field , cy test.

* fix: unit test

* fix: test connection service type issue

* Added powerbi docs link in req

* Add info on hive

* Remove unused markdowns

* Add req for datalake

* add hive requirements header

* Snowflake & Biguqery

* Update Mssql and Postgres

* mysql and amundsen requirements updated

* Update Mssql and Postgres

* added username

* chore: fix cy expression issue

* chore: reset active field state on step change.

* fix: affix target container issue

* fix: unit test

* fix: cypress for postgres and glue

---------

Co-authored-by: Milan Bariya <52292922+MilanBariya@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: NiharDoshi99 <51595473+NiharDoshi99@users.noreply.github.com>
Co-authored-by: Milan Bariya <milanbariya12@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
2023-03-29 14:18:17 +05:30
Onkar Ravgan
5d6e18dc28
Fix 10642: Mark delete entities and tags toggle (#10695)
* Added mark delete logic

* Final test and optimization

* After merge fixes

* Added include tags for dash pipelines dbt

* added docs and fixed test

* Fixed py tests

* Added UI changes for following newly added fields:
- markDeletedDashboards
- markDeletedMlModels
- markDeletedPipelines
- markDeletedTopics
- includeTags

* Fixed failing unit tests

* updated json files of localization for other languages

* Improved localization changes

* added localization changes for other languages

* Updated mark deleted desc

* updated the ingestion fields descriptions in the ingestion form for UI

* automated localization changes for other languages

* updated descriptions for includeTags field for dbtPipeline and databaseServiceMetadataPipeline json

* fixed issue where includeTags field was being sent in the dbtConfigSource

* Added flow to input taxonomy while adding BigQuery service.

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2023-03-29 12:41:44 +05:30
Milan Bariya
3d1bbb1037
Add a configurationto skip Snowflake transient and tmp tables (#10665)
* Add a configurationto skip  Snowflake transient and tmp tables

* Fix Python checkstyle

* add separate query for transient tables

* Move skipTempTables into SnowflakeConnection

* Fix merge conflict

* change skip word to include

* Add title in json file

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-28 06:29:59 +02:00
Mohit Yadav
29119dda89
Add docs for SAMl and AWS saml setup (#10787) 2023-03-28 01:05:21 +05:30