Shiyang Xiao 241f3c68d7
Fixes #14413: Add SAS connector (#14415)
* feat: SAS Viya connector

* refactor SASCatalog to SAS

* add SAS logo to UI and connection documentation

* doc changes

* modify ingestion logic

* revert original changes

* added support for dataflow & perfect logic for reports/datatables

* add filter doc

* more updates to perfect ingestion for each asset type

* fix a bug with table lineage not created properly

* Delete ingestion/pipelines/sasCatalog.yaml

* precomit fix

* Conversion to database connector

* minor fixes

* make custom properties type generic

* Add SAS javaEnum

* add dummy variable for sas.yaml

---------

Co-authored-by: lizmc <liz.mcintosh@sas.com>
Co-authored-by: Shiyang Xiao <Shiyang.Xiao@sas.com>
2024-01-11 06:46:57 -08:00

2.8 KiB

title slug
Run the SAS Connector Externally /connectors/database/sas/yaml

Run the SAS Connector Externally

In this section, we provide guides and references to use the SAS connector.

Configure and schedule SAS metadata workflows from the OpenMetadata UI:

{% partial file="/v1.2/connectors/external-ingestion-deployment.md" /%}

Requirements

{%inlineCallout icon="description" bold="OpenMetadata 1.3.0 or later" href="/deployment"%} To deploy OpenMetadata, check the Deployment guides. {%/inlineCallout%}

Python Requirements

To run the SAS ingestion, you will need to install:

pip3 install "openmetadata-ingestion[sas]"

Metadata Ingestion

All connectors are defined as JSON Schemas. Here you can find the structure to create a connection to SAS.

In order to create and run a Metadata Ingestion workflow, we will follow the steps to create a YAML configuration able to connect to the source, process the Entities if needed, and reach the OpenMetadata server.

The workflow is modeled around the following JSON Schema

1. Define the YAML Config

{% codePreview %}

{% codeInfoContainer %}

Source Configuration - Service Connection

{% codeInfo srNumber=12 %}

serverHost: Host and port of the SAS Viya deployment.

{% /codeInfo %}

{% codeInfo srNumber=13 %}

username: Username to connect to SAS Viya. This user should have privileges to read all the metadata in SAS Information Catalog.

{% /codeInfo %}

{% codeInfo srNumber=14 %}

password: Password to connect to SAS Viya.

{% /codeInfo %}

Sink Configuration

{% codeInfo srNumber=18 %}

To send the metadata to OpenMetadata, it needs to be specified as type: metadata-rest.

{% /codeInfo %}

{% partial file="/v1.2/connectors/yaml/workflow-config-def.md" /%} {% /codeInfoContainer %}

{% codeBlock fileName="filename.yaml" %}

source:
  type: SAS
  serviceName: local_sas
  serviceConnection:
    config:
      type: SAS
      serverHost: http://localhost:10000
      username: username
      password: password

sourceConfig: config: type: DatabaseMetadata

```yaml {% srNumber=18 %}
sink:
  type: metadata-rest
  config: {}

{% partial file="/v1.2/connectors/yaml/workflow-config.md" /%}

{% /codeBlock %}

{% /codePreview %}

{% partial file="/v1.2/connectors/yaml/ingestion-cli.md" /%}