213 Commits

Author SHA1 Message Date
Prajwal214
5a1753aa2f
Doc: Updating Database Connector Docs for SSL (#17269)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-08-07 11:41:00 +05:30
Onkar Ravgan
79348a2259
MINOR: Added Alation Sink connector (#16982) 2024-07-24 12:08:07 +05:30
Suman Maharana
d3ea1ead01
Fix #12677: Added Synapse Connector - docs and side docs (#17041) 2024-07-22 12:35:46 +05:30
Prajwal214
59d9a0299c
Doc: Updated Doris & Redshift Docs (#17123)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-07-22 06:42:54 +00:00
Prajwal214
b27ce3f4f3
Doc: Updating Service Connection Images (#17058)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-07-18 10:39:57 +05:30
Prajwal214
19c43273dc
Doc: Update Custom Connector Docs (#17002)
* Doc: Update Custom Connector Docs

* Doc: Update Custom Connector Docs

---------

Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-07-12 10:33:47 +02:00
Prajwal214
f5abfdacfe
Doc: Updating Webhooks Docs (#16915)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-07-04 13:43:05 +05:30
Prajwal214
a9f6bd4ed1
Doc: Adding how-to guide for domains & Data Products (#16778)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-07-04 12:37:38 +05:30
Pere Miquel Brull
e3bbb18a03
DOCS - Storage Connector example (#16910) 2024-07-03 11:55:27 +02:00
Prajwal214
de7a14bf8f
Doc: Updating old reference Images of Ingestion Bot (#16843)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
2024-07-02 10:35:54 +05:30
k.nakagaki
9a31a35296
Fixes 9875: supporting gcp secret manager (#16505)
* Split ExternalSecretsManagerTest to new ExternalSecretsManagerTest and AWSBasedSecretsManagerTest

* implement SecretsManagerFactory to create GCPSecretsManager

* implement GCPSecretsManager

* implements gcp secret manager.

* Fix it for the GCP's rule.

* create a template of GCP

* fix compile error

* implements to use project_id

* add library for the google cloud secret manager

* add test code for using google credential in the docker container

* modify docker-compose.yml for GCP

* add google_crc32c module

* modify ways to get project id

* create a new docker-compose.yml for Google Cloud

* create a new document

* create compose file for gcp secret manager

* fix invalid styles and formats

* downgrade google library to avoid conflicting protoc versions
2024-06-28 21:09:02 -07:00
Pere Miquel Brull
c7220ccdc9
DOCS - Fix storage YAML & prep 1.5 docs (#16833) 2024-06-28 07:49:30 +02:00
harshsoni2024
afec7703cc
Fix #11888: Apache flink connector (#16601) 2024-06-26 19:21:16 +05:30
Chirag Madlani
e02ead8133
add connector teradata icon (#16803) 2024-06-26 12:48:23 +05:30
Mayur Singal
d8a4e4d4ff
MINOR: Add docs for big query create credentials/service account (#16759) 2024-06-21 15:58:40 +05:30
gpby
db959c33a6
Add Teradata connector docs (#16672)
* Add Teradata doc to ui

* Add Teradata doc
2024-06-17 17:47:04 -07:00
Prajwal214
1f27bb7feb
Doc: Adding How-to Guide for Incident Manager (#16674)
* Doc: Adding Docs for Incident Manager

* Doc: Adding Docs for Incident Manager

---------

Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
Co-authored-by: Shilpa Vernekar <94032785+ShilpaVernekar@users.noreply.github.com>
2024-06-17 09:28:50 -07:00
harshsoni2024
b87956cbf0
match all params in metadata ingestion docs-ui-yaml (#16528) 2024-06-05 11:05:24 +05:30
Prajwal214
7959e2502e
Updating S3 Connection Reference (#16391) 2024-05-24 14:55:46 +05:30
Onkar Ravgan
b7ad0ee010
MINOR: [DOCS] Added pbit lineage docs (#16388)
* Added pbit docs

* add dbservicenames field

* format doc

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-05-22 19:04:33 +05:30
Suman Maharana
60f4963759
MINOR: Cleaned OpenLineage Docs (#16339)
* cleaned openlineage docs

* cleaned openlineage docs

* cleaned openlineage docs
2024-05-18 11:08:25 +02:00
Teddy
311243a828
DOC - Updated Quality and Observability (#16330)
* doc: update 1.4 DQ & Observability doc

* doc: Added observabiliuty and quality doc
2024-05-17 15:30:40 +02:00
Onkar Ravgan
93470b7a43
MINOR: Updated Alation connector 1.4 docs (#16327)
* fixed alation 1.4 docs

* fix docs format

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-05-17 17:43:53 +05:30
Pere Miquel Brull
c24b4e0178
DOCS - Publish 1.4 docs & Remove 1.1 and 1.2 (#16308) 2024-05-16 16:10:54 +02:00
Mayur Singal
b1b737b837
MINOR: Export Lineage Docs (#16289) 2024-05-16 16:04:55 +05:30
Imri Paran
f060d60c1b
MINOR: Added docs for failed rows sample in data quality (#16218)
* Added failed row sample docs

* title
2024-05-10 12:45:00 +02:00
Sachin Chaurasiya
e24e7cfd4b
doc: update custom property doc (#16213) 2024-05-10 08:03:02 +05:30
Prajwal214
78a53f2a9b
Minor: Updated the IntelliJ Dependency For Openmetadata-UI (#16179) 2024-05-09 16:59:09 +05:30
Prajwal214
43c205ae00
Minor: Updated the document for Airflow DAGs Deployed as Examples (#16182) 2024-05-09 16:58:02 +05:30
Suman Maharana
6915c1a1b6
Fix #15090 [1.4] Added dbt existing glossary and tier ingestion (#15833)
* Added dbt glossary and tier ingestion

* changed elif to if condition:pylint

* Added dbtMeta model for glossary and tier

* removed source

* Optimization: Reused the dtb_tags

* pylint format

* py_format

* Removed unnecessary models and minor changes

* Removed empty string assignment
2024-04-17 17:10:20 +02:00
Onkar Ravgan
370abda43f
dbt docs revamp (#15787) 2024-04-03 20:42:30 +05:30
Shilpa Vernekar
f28214e0c0
Missing docs added (#15763) 2024-04-03 09:03:08 +00:00
Harsh Vador
1db15dc840
Change connectors images format to webp (#15781)
* Change connectors images format to webp

* compress connectors images
2024-04-03 11:50:50 +05:30
Suman Maharana
a8d288cc32
Fix #11351 Added Kafka Connect connector docs (#15747)
* Added Kafka Connect JSON Schemas

* Added Kafka Connect Documentation

* Revert "Added Kafka Connect JSON Schemas"

This reverts commit 89cf700e93f1391f24156a1235e5d5e70b067704.

* Added webp image formats for kafkaconnect docs

---------

Co-authored-by: Suman Maharana <suman@Reyna.local>
2024-04-01 11:34:38 +05:30
harshsoni2024
884029f031
Fix #14089: Add QlikCloud connector support (#15610) 2024-03-28 20:15:22 +05:30
IceS2
ce3f124a33
MINOR: Add Multithread Documentation (#15706)
* Add Multithread Documentation

* Add general considerations
2024-03-27 08:30:16 +01:00
Aniket Katkar
dcbb1dd0f8
Docs: Fix the wrong image references for connector details pages (#15559)
* fix the wrong image references for connector details pages

* Add ADLS and GCS Pages

* Add adls and gcs logos

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2024-03-14 11:00:31 +05:30
Ayush Shah
1c2fbdd9f4
MINOR: Add / Fix GCS and ADLS - docs, bugs (#15502)
Add GCS and ADLS docs
2024-03-12 21:13:24 +05:30
mgorsk1
98850ab5cc
feat: OpenLineage integration (#15317)
* 🎉 Init OpenLineage connector

Co-authored-by: dechoma <dominik.choma@gmail.com>

* MLH - make linter happy

* review fixes

* 🐛 Fix path for ol event in tests

* 🐛 Fix path for ol event in tests

* Update ingestion/setup.py

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>

* Update ingestion/src/metadata/ingestion/source/pipeline/openlineage/metadata.py

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>

* Update ingestion/src/metadata/ingestion/source/pipeline/openlineage/models.py

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>

* review fixes 2

* linter

* review

* review

* make linter happy

* fix test_yield_pipeline_lineage_details test

* make linter happy

* fix tests

* fix tests 2

---------

Co-authored-by: dechoma <dominik.choma@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2024-03-12 08:39:25 +01:00
Imri Paran
67c57ec4f0
docs: 1.4.x-SNAPSHOT (#15506)
* docs: 1.4.x-SNAPSHOT
added partials
added images
updated paths
2024-03-11 09:42:26 +00:00
Aniket Katkar
6a0750eb0c
Docs: Update connectors in the documentation (#15384)
* update the connectors lists in documentation pages

* add missing connector images
2024-02-28 08:14:11 +00:00
Imri Paran
7036b3cbf7
MINOR: BigTable connector docs (#15145)
* feat(connector): add BigTable

* bigtable work

1. docstrings
2. tests
3. created a Row BaseModel
4. implemented a ClassConverter

* docs moved to separate PR

* format files

* docs(bigtable): added docs

* fix after merge
2024-02-19 08:11:29 +01:00
Mayur Singal
e6c2c36f11
MINOR: Add databricks configuration docs for spark agent (#15182) 2024-02-14 21:16:34 +05:30
Pere Miquel Brull
eb501c1996
MINOR - Breaking change for alerts docs (#15026) 2024-02-05 11:54:59 +01:00
Ayush Shah
6838fadec6
MINOR: Fix Docs ( Security, DBT ) (#14980) 2024-02-01 13:08:48 +05:30
IceS2
373cafcda2
Fixes #5448: Implement initial Iceberg Connector using PyIceberg (#14825)
* Create the iceberg connection schema

* Link the IcebergConnection configuration with the forms on the UI

* Add the pyiceberg dependency on the ingestion package

* Create the get_connection and test_connection functions

* First iteration on the iceberg ingestion logic

* Add A more comprehensive implementation of the Iceberg Source

* Add UnitTests

* Update icebergConnection definition

* Update the iceberg souce code based on new schema

* Updated icebergConnecgtion schema for simplicity and to be able to configure Converters

* Updated setup dependencies to be more flexible

* Updated get_owner_ref logic

* Fix formatting

* Changed the icebergConnection json schema structure to enable the ClassConverters

* Add the IcebergCatalog and IcebergFileSystem ClassConverters

* Refactor the code to take into account the new jsonSchema structure

* Fix formatting

* Add Documentation for the Iceberg Connector

* Fix Menu order for Iceberg

* ui: add Iceberg service icon and constant

* Fix DynamoDb Catalog issue due to how PyIceberg instantes it

* Changed uri title to URI

* Fix ClassConverter for Iceberg

* Fix GetSecretValue for password types

* Fix formatting

* Fix formatting

* Add Iceberg Connector Images for the docs

* Add pylint disable for Hacky super() call

* Add Iceberg.md for the UI docs

* Fix pylint complaint

* Fix pylint complaint

* Fix UnitTests

* fix type error and unit tests

* update pipeline type checks

* Fix Sonar Cloud complaints

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-01-29 06:32:58 +01:00
Shiyang Xiao
9f5a70bd71
MINOR - update docs & added unit test for SAS Connector (#14743)
Co-authored-by: Shiyang Xiao <Shiyang.Xiao@sas.com>
2024-01-23 14:55:29 -08:00
Shilpa Vernekar
d7f4172416
DOCS: Knowledge Center Article and other changes (#14795) 2024-01-19 10:44:05 +00:00
Shiyang Xiao
241f3c68d7
Fixes #14413: Add SAS connector (#14415)
* feat: SAS Viya connector

* refactor SASCatalog to SAS

* add SAS logo to UI and connection documentation

* doc changes

* modify ingestion logic

* revert original changes

* added support for dataflow & perfect logic for reports/datatables

* add filter doc

* more updates to perfect ingestion for each asset type

* fix a bug with table lineage not created properly

* Delete ingestion/pipelines/sasCatalog.yaml

* precomit fix

* Conversion to database connector

* minor fixes

* make custom properties type generic

* Add SAS javaEnum

* add dummy variable for sas.yaml

---------

Co-authored-by: lizmc <liz.mcintosh@sas.com>
Co-authored-by: Shiyang Xiao <Shiyang.Xiao@sas.com>
2024-01-11 06:46:57 -08:00
Sriharsha Chintalapani
182a19f3cc
Add docs on where to get the API token (#14633) 2024-01-09 07:55:40 -08:00