3975 Commits

Author SHA1 Message Date
Tamas Nemeth
63bc830cfe
Data domain containers ingestion (#4051) 2022-02-07 09:51:49 -08:00
Danilo Peixoto
52272d6561
feat(cli): add --force option to ingest rollback subcommand (#4032) 2022-02-06 14:49:10 -08:00
John Joyce
d19241740e
refactor(model): refactor new Assertion models (#4064) 2022-02-06 14:30:40 -08:00
Aditya Radhakrishnan
61db4ed152
refactor(ingest) - remove snowflake_common dependency on aws_common (#4054)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-04 23:24:20 -08:00
Kevin Hu
1a2f75b1e6
fix(ingest): data-lake - add aws dependencies (#4060) 2022-02-04 21:12:09 -08:00
Kevin Hu
6fe062f37c
feat(ingest): log CLI invocations and completions (#4062) 2022-02-04 21:11:04 -08:00
Pedro Silva
ef617784b1
Mark data lake metadata source as Beta (#4061) 2022-02-04 16:27:44 -08:00
Tamas Nemeth
cc32c30b2a
fix(ingest): bigquery - fix for hitting limit if there are too many partitioned tables (#4056) 2022-02-04 14:02:36 -08:00
Swaroop Jagadish
aab622793d
feat(ci): pin tox requirements to speed up ci runs, remove airflow-1 suite until we can pin it (#4055) 2022-02-04 07:32:38 -08:00
ksrinath
4c24f386a6
feat(model): data quality model (#3787)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Mayuri N <mayuri.nehate@gslab.com>
2022-02-02 22:56:08 -08:00
Swaroop Jagadish
ded16809da
feat(ingest): add tests for platform instance (#4047) 2022-02-02 22:52:50 -08:00
John Joyce
169cd4fde9
feat(containers): Adding Containers UI (as demo'd in Jan Townhall) (#4037) 2022-02-02 13:51:39 -08:00
Ravindra Lanka
f20382f956
feat(ingest): framework - client side changes for monitoring and reporting (#3807) 2022-02-02 13:19:15 -08:00
Ravindra Lanka
f4209504f1
feat(ingest): support Kafka confluent external schema resolution by name or subject (#4035) 2022-02-02 07:44:56 -08:00
John Joyce
7fef9a514a
feat(containers): Backend for Asset Containers (as demo'd in townhall) (#4019) 2022-02-01 11:07:34 -08:00
Tamas Nemeth
928ab74f33
feat(ingest): bigquery - profile only the latest partition/shard (#3930) 2022-02-01 10:05:35 -08:00
mayurinehate
1afe8876b7
feat(ingest): nifi - handle provenance api variation for older versions (#4022) 2022-02-01 10:03:05 -08:00
Tamas Nemeth
771c8567da
fix(ingest): snowflake - Run authentication validation if default value used (#4024) 2022-02-01 10:01:29 -08:00
Tamas Nemeth
68711222d4
feat(ingest): usage-stats - add ability to ignore users from top users calculation (#3735) 2022-02-01 00:11:23 -08:00
mayurinehate
faac911df8
feat(ingest): use trino python client with sqlalchemy, supports python3.6 (#3888) 2022-01-31 09:22:55 -08:00
Swaroop Jagadish
e29d2cbe7e
fix(ingest): adding platform instance urn to data platform instance aspects (#4015) 2022-01-31 08:49:03 -08:00
Swaroop Jagadish
c27f1f9487
fix(ingest): lint fix a few files (#4016) 2022-01-30 23:32:29 -08:00
Tamas Nemeth
646dd6d5f0
fix(ingest): profiler - enable profiling for low cardinality number columns (#3990) 2022-01-30 17:36:18 -08:00
Aseem Bansal
f32643148e
fix(docs): fix business glossary docs (#3916) 2022-01-30 17:25:11 -08:00
Aseem Bansal
7180ae5ff6
fix(ci): fix linting lib version requirement (#3939) 2022-01-30 17:20:46 -08:00
Kevin Hu
b84294804a
feat(ingest): emit flags and stats for profiling telemetry (#3969) 2022-01-30 17:19:00 -08:00
Aditya Radhakrishnan
b5ff558f03
feat(model) - Update FabricType enum to represent more fabrics (#3997)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-30 18:19:03 -06:00
Michael A. Schlosser
c36662f837
feat(ingest): snowflake - support for additional auth mechanisms (#4009) 2022-01-30 11:47:53 -08:00
John Joyce
3c2a4fe39d
Refining docs (#4001) 2022-01-28 15:00:35 -08:00
Aseem Bansal
d3b7cece7a
fix(build): m1 - harden pre-flight script for M1 (#3958) 2022-01-27 23:01:18 -08:00
Aseem Bansal
308dbf2b1a
fix(ingest): superset - fix connection inference for redshift (#3944) 2022-01-27 22:58:34 -08:00
RyanHolstien
f6cbeba515
fix(cli): add missing classes to typed aspect map (#4000)
Co-authored-by: Ryan Holstien <ryan@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-27 22:55:40 -08:00
Nick Wu
149d2fa3d0
docs(ingest): add missing back tick for metadata-ingestion/README.md (#4003) 2022-01-27 22:51:48 -08:00
John Joyce
27c543a3aa
feat(domains): UI for Asset Domains (p2) (#3995) 2022-01-27 22:02:41 -08:00
Swaroop Jagadish
9a9f448ebf
feat(ingest): platform-instance - changes to support platform instances in sources (#3996) 2022-01-27 15:31:25 -08:00
dipeshmaurya
ad0019cb22
fix(docs): typo in bigquery_to_datahub.yml (#3988) 2022-01-27 15:01:59 -08:00
John Joyce
282851eeed
feat(domains): Adding backend for Asset Domains (p1) (#3952) 2022-01-27 13:07:11 -08:00
John Joyce
271784c9c1
feat(ui): UI-based ingestion (as featured in Dec Townhall) (#3975) 2022-01-27 10:33:12 -08:00
John Joyce
9366a47f88
fix(quickstart): Assign correct mysql-setup container for M1 and remove "head" default version. (#3987)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-26 18:15:54 -08:00
John Joyce
cf38b61a7f
feat(ingest): looker - optional external_base_url config for chart and dashboard urls (#3985)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-26 15:06:34 -08:00
Kevin Hu
62f7d05557
feat(ingest): data-lake - support for Avro files (#3913) 2022-01-26 10:29:00 -08:00
Harshal Sheth
e6060c03be
fix(ingest): don't dump variables for config errors (#3974) 2022-01-25 22:30:18 -08:00
Harshal Sheth
ea5a66bbc5
fix(ingest): serialization - tighten conditions for restli json transformation (#3973) 2022-01-25 21:56:04 -08:00
Tamas Nemeth
cec541d827
fix(ci): ingestion - Restrict python to <=3.9.9 due to upstream looker sdk issues (#3961)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-25 19:11:53 -08:00
Aseem Bansal
b6a340d46b
fix(example): fix typo (#3907) 2022-01-25 19:06:58 -08:00
Harshal Sheth
5168f0a50e
refactor(ingestion): remove duplicate aspect type (#3972) 2022-01-25 19:05:14 -08:00
iasoon
5dac30c93e
fix(ingestion): match default username for Azure OIDC and Azure ingestion source (#3926) 2022-01-25 18:57:18 -08:00
John Joyce
0da1a6b53d
fix(graphQL): Fixing set ordering in batchGet of entities (#3950) 2022-01-23 11:24:09 -08:00
Ravindra Lanka
715a2e5f8e
feat(ingest): snowflake, glue - add support for lineage to s3 (#3946) 2022-01-21 17:19:21 -08:00
Ravindra Lanka
7eda2bf144
fix(ingest): snowflake_usage - emit usage workunits when include_operational_stats is enabled. (#3949) 2022-01-21 17:16:03 -08:00