12076 Commits

Author SHA1 Message Date
vijayan-nallasami-curve
26329e0af9
added node_type_pattern in dbt yaml file (#2705) 2021-06-17 10:24:33 -07:00
vijayan-nallasami-curve
943c54a835
Removed key not in catalog filter from dbt and added node filter using AllowDenyPattern (#2702) 2021-06-17 10:24:12 -07:00
Harshal Sheth
5d93f249b4
feat(ingest): expose additional types to Python via codegen (#2712) 2021-06-17 10:04:28 -07:00
kuntalkumarbasu
084987408c
feat: Adding annotation option to frontend service (#2674) 2021-06-16 17:00:38 -07:00
Harshal Sheth
7e9a04479b
test(ingest): simplify docker cleanup commands (#2699) 2021-06-16 16:59:28 -07:00
Harshal Sheth
26dcece8ec
fix(ingest): use looker data platform (#2708) 2021-06-16 16:58:13 -07:00
Kevin Hu
63fe82995b
feat(ingest): Add test case and docs for SQL view ingestion (#2709) 2021-06-16 16:51:57 -07:00
Brian
a5f9b8dfe9
feat(entities): add markdown description update/viewer feature in dataset, datajob, dataflow, chart and dashboard, update ui/ux (#2707) 2021-06-16 15:48:27 -07:00
Gabe Lyons
523c3bf1d4
feat(aspects): support fetching of versioned aspects (#2677) 2021-06-16 10:03:21 -07:00
Peter Mortier
a0579321b3
Add support for different oidc client authentication methods (#2691) 2021-06-16 09:19:47 -07:00
JeffSkinner
5b595b4c9e
feat(gms): support basic auth header when connecting to elasticSearch (#2701) 2021-06-16 09:11:02 -07:00
Dexter Lee
f3f562fe08
fix(NoCode): Fix product analytics (#2697) 2021-06-15 09:42:16 -07:00
John Joyce
cc95916201
feat(gms): Merge MAE, MCE consumers into GMS (#2690) 2021-06-15 08:44:15 -07:00
John Joyce
7591c8994a
feat(datahub cli): DataHub CLI Quickstart (#2689) 2021-06-14 17:15:24 -07:00
Kevin Hu
1c364b52f6
feat(docs): Docs for S3 ingestion with AWS Glue (#2672) 2021-06-14 17:08:50 -07:00
Remi
9fb2e6aaa8
Only work with dbt catalog data if load_catalog is False (#2686) 2021-06-14 16:58:19 -07:00
Gabe Lyons
0b75b4a96a
removing whitespace from service aspect (#2688) 2021-06-12 17:48:36 -07:00
Vincenzo Lavorini
a7fc76f590
feat(sql_views): added views as datasets for SQLAlchemy DBs (#2663) 2021-06-11 17:30:33 -07:00
Harshal Sheth
1b539220d5
feat(ingest): support Oracle service names (#2676) 2021-06-11 17:27:34 -07:00
Harshal Sheth
1857f85242
fix(ingest): upgrade acryl-pyhive to use sasl3 instead of sasl (#2684) 2021-06-11 17:24:41 -07:00
Rickard Cardell
2cb06188ce
Fix 2592 neo4j connection options (#2632) 2021-06-11 09:55:30 -07:00
Harshal Sheth
5eee818a61
fix(ingest): pin to new mypy version (#2670) 2021-06-11 09:44:54 -07:00
Kevin Hu
24268c2021
feat(ingest): headers for codegen Python scripts (#2637) 2021-06-11 09:44:18 -07:00
Thomas Larsson
61a8994bb6
fix(analytics): Support sasl authentication to kafka (#2675) 2021-06-10 12:10:13 -07:00
zack3241
18557ed42a
feat(k8s): Add imagePullSecrets to all K8's jobs (#2671) 2021-06-10 12:06:27 -07:00
zack3241
91eb3cc57e
Add get_identifier to hive source in metadata ingestion (#2667) 2021-06-09 15:12:17 -07:00
Gabe Lyons
28b5856c36
fix(no-code): Adding Chart input relationship annotations (#2669)
* chart input relationship pdl fix

* commiting schema.avsc changes
2021-06-09 15:09:50 -07:00
Kevin Hu
ebdaa0e359
feat(ingest): Feast ingestion integration (#2605)
* Add feast testing setup

* Init Feast test script

* Add feast to dependencies

* Update feast descriptors

* Sort integrations

* Working feast pytest

* Clean up feast docker-compose file

* Expand Feast tests

* Setup feast classes

* Add continuous and bytes data to feature types

* Update field type mapping

* Add PDLs

* Add MLFeatureSetUrn.java

* Comment out feast setup

* Add snapshot file and update inits

* Init Feast golden files generation

* Clean up Feast ingest

* Feast testing comments

* Yield Feature snapshots

* Fix Feature URN naming

* Update feast MCE

* Update Feature URN prefix

* Add MLEntity

* Update golden files with entities

* Specify feast sources

* Add feast source configs

* Working feast docker ingestion

* List entities and features before adding tables

* Add featureset names

* Remove unused

* Rename feast image

* Update README

* Add env to feast URNs

* Fix URN naming

* Remove redundant URN names

* Fix enum backcompatibility

* Move feast testing to docker

* Move URN generators to mce_builder

* Add source for features

* Switch TypeClass -> enum_type

* Rename source -> sourceDataset

* Add local Feast ingest image builds

* Rename Entity -> MLPrimaryKey

* Restore features and keys for each featureset

* Do not json encode source configs

* Remove old source properties from feature sets

* Regenerate golden file

* Fix race condition with Feast tests

* Exclude unknown source

* Update feature datatype enum

* Update README and fix typos

* Fix Entity typo

* Fix path to local docker image

* Specify feast config and version

* Fix feast env variables

* PR fixes

* Refactor feast ingest constants

* Make feature sources optional for back-compatibility

* Remove unused GCP files

* adding docker publish workflow

* Simplify name+namespace in PrimaryKeys

* adding docker publish workflow

* debug

* final attempt

* final final attempt

* final final final commit

* Switch to published ingestion image

* Update name and namespace in java files

* Rename FeatureSet -> FeatureTable

* Regenerate codegen

* Fix initial generation errors

* Update snapshot jsons

* Regenerated schemas

* Fix URN formats

* Revise builds

* Clean up feast URN builders

* Fix naming typos

* Fix Feature Set -> Feature Table

* Fix comments

* PR fixes

* All you need is Urn

* Regenerate snapshots and update validation

* Add UNKNOWN data type

* URNs for source types

* Add note on docker requirement

* Fix typo

* Reorder aspect unions

* Refactor feast ingest functions

* Update snapshot jsons

* Rebuild

Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-09 15:07:04 -07:00
Kevin Hu
c0ace2ce59
fix(ingest): fix MyPy stubs (#2666) 2021-06-08 16:10:16 -07:00
Harshal Sheth
2f7b4f9def
build(docker): test docker builds in pull request CI (#2661) 2021-06-08 09:47:32 -07:00
Dexter Lee
83c5259d14
docs(nocode): fix typo in autocomplete annotation (#2664) 2021-06-08 09:08:45 -07:00
Gabe Lyons
d3a1e2ccab
docs(nocode): adding documentation to handle backwards incompatibility issues (#2662) 2021-06-07 23:20:00 -07:00
Harshal Sheth
2123c8b6d7
fix(ingest): exclude mssql-odbc from "all" extra (#2660) 2021-06-07 14:00:35 -07:00
Dexter Lee
9c030d8325
fix(NoCode): Update snapshot json to latest (#2655) 2021-06-07 11:52:52 -07:00
Harshal Sheth
31eae24300
fix(ingest): support mssql encryption via ODBC (#2657) 2021-06-04 18:19:11 -07:00
Harshal Sheth
051fa253d1
fix(docker): use debug tag in local dev images (#2658) 2021-06-04 18:12:26 -07:00
Harshal Sheth
e73c6525e9
revert: "fix(docker): pin containers to golden hash for release (#2654)" (#2659)
This reverts commit a483933eab6d072ba6d2a414e378eee1e80dbdc6 and moves
us back to using HEAD in quickstart on the master branch.
2021-06-04 18:11:36 -07:00
Gabe Lyons
b1ff56f606
docs(nocode): Adding documentation for no-migration upgrade option (#2656) 2021-06-04 17:54:17 -07:00
Dexter Lee
ee454eb5d6
feat(nocode): Add datahub-upgrade job to helm chart and set version to v0.8.1 (#2643) 2021-06-04 15:13:32 -07:00
Harshal Sheth
a483933eab
fix(docker): pin containers to golden hash for release (#2654) v0.8.1 2021-06-04 12:46:57 -07:00
John Joyce
9829576202
feat(datahub-upgrade): improve no code upgrade logging (#2653) 2021-06-04 12:21:17 -07:00
Harshal Sheth
f91062d827
docs(nocode): fix links to datahub-upgrade (#2651) 2021-06-04 11:30:06 -07:00
John Joyce
d26903f160
fix(gms): Return empty snapshots when one does not exist (#2646) 2021-06-04 11:28:53 -07:00
John Joyce
cc3caa998b
fix(datahub-upgrade): fixing mis-spelled schema registry url (#2652) 2021-06-04 11:05:13 -07:00
Harshal Sheth
9e79272a5f
fix: use head tag by default in quickstart (#2650) 2021-06-04 09:52:10 -07:00
Dexter Lee
e187b07cf3
docs(nocode): Update migration docs with workaround for stuck upgrade(#2649) 2021-06-04 00:18:41 -07:00
Dexter Lee
d5934f70bb
fix(nocode): Fix docker image tag for run_upgrade script (#2648) 2021-06-03 21:42:23 -07:00
Dexter Lee
5ca484a153
fix(analytics): fix index names (#2647) 2021-06-03 21:38:43 -07:00
Harshal Sheth
7f0c6bb5c9
fix(build): don't purge ingestion codegen files on gradle clean (#2645) 2021-06-03 20:33:25 -07:00
John Joyce
c98446c9a5
fix(datahub-upgrade): remove debug flag (#2640) 2021-06-03 16:26:20 -07:00