3904 Commits

Author SHA1 Message Date
Mayuri Nehate
6c42064332
fix(ingest): snowflake - fix missing snowflake lineage when table_pattern is set (#6410) 2022-11-11 16:47:09 +01:00
Mayuri Nehate
1b739319f0
fix(ingest/snowflake): fix TypeError when columns are returned as None in snowflake query (#6404) 2022-11-10 12:02:49 -08:00
Harshal Sheth
ae2ea524a4
fix(ingest): use profiler config settings correctly (#6354)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-09 19:03:22 -08:00
Harshal Sheth
96f782730e
feat(ingest/tableau): re-authenticate if the token expires (#6380)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-09 18:55:24 -08:00
Tamas Nemeth
ef5c7128da
fix(ingest): unity - Removing unneeded dependency for sqlalchemy from unity-catalog to fix connector setup (#6379) 2022-11-09 15:57:17 +01:00
Marcin Szymański
44cfc29af0
fix(ingest): unity - add missing date type (#6385) 2022-11-09 15:56:22 +01:00
Harshal Sheth
bfb682ace7
build(ingest): pin mypy version (#6391) 2022-11-09 11:38:42 +01:00
wangsaisai
92192d2410
feat(ingest): enable container stateful ingestion (#6343) 2022-11-08 12:34:43 +01:00
Harshal Sheth
1a81d8de6a
feat(ingest): supports MCEs in domain transformer (#6364) 2022-11-05 11:41:43 -07:00
Tamas Nemeth
07a0e86470
fix(ingest): bigquery - Sending lineage aspects as patch (#6313)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-11-04 09:52:34 -07:00
Harshal Sheth
3080b9e3ee
fix(ingest): only log vars if requested (#6362) 2022-11-03 20:16:07 -07:00
Chris Collins
f24f74598b
fix(ui): Handle encoded schemaField urns on the frontend (#6321) 2022-11-03 13:43:10 -07:00
cburroughs
39c84c2f5b
feat(ingest): upgrade feast (#6186)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-03 11:23:19 -07:00
Harshal Sheth
ab904cdd37
fix(ingest): fix log line interpolation (#6349) 2022-11-03 10:56:56 -07:00
Harshal Sheth
0ca3383d30
feat(ingest): support reserved keywords in model codegen (#6351) 2022-11-02 22:26:56 -07:00
Harshal Sheth
e0c235a0c1
doc(ingest): clarify that redshift-usage does not support column-level usage (#6347) 2022-11-02 16:27:58 -07:00
Mayuri Nehate
c01aa53aa7
fix(ingest): fix schema field urns with empty field path (#6338) 2022-11-02 15:02:13 +01:00
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x (#6331) 2022-11-01 21:12:34 -07:00
Oleksandr
0a0cf70b6f
feat(ingest): power-bi - add support for on-prem PowerBI Report Server (#5369) 2022-11-01 16:05:42 -07:00
mohdsiddique
fbb1b7271c
feat(ingestion): business-glossary - Add values and relatedTerms support (#6148)
* WIP

* updated snapshot

* merge

* while creating BusinessGlossaryConfig object, the version field always get set to None

* WIP

* lint fix

* remove field

* doc update

* rename the fields

* review comments

* lintFix

* Fix the business glossary example link

* small fixes to examples and docs

* some small tweaks to the model annotations

Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-11-01 15:17:30 -07:00
John Joyce
3ff49d4499
refactor(bootstrap): Adding assertions data to bootstrap (#6324) 2022-11-01 12:46:04 -07:00
Mayuri Nehate
8e1deb6853
fix(ingest): looker - fix exception logging (#6319) 2022-10-31 09:42:06 -07:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00
Kevin Neville
3eae087a30
chore(ci): update actions versions addressing warnings in builds. (#6308) 2022-10-28 17:10:30 -07:00
David Schmidt
5934a0e79f
fix(docs): fix custom transformer example interface (#6309) 2022-10-28 17:00:18 -07:00
Harshal Sheth
ef824bd082
feat(ingest): add fallthrough support to KeyValuePattern (#6302) 2022-10-28 11:07:47 +02:00
Harshal Sheth
5b4f5f8d26
feat(ingest): mssql - add type mappings for BIT and SQL_VARIANT (#6304) 2022-10-28 11:05:06 +02:00
Harshal Sheth
d986a044fb
fix(ingest): tableau - add extra debug info for lineage failure (#6285) 2022-10-26 20:01:55 -07:00
Harshal Sheth
e27b017427
fix(ingest): dbt - lowercase external urns + cleanup config (#6289)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 19:46:51 -07:00
Mayuri Nehate
29b99052d7
feat(ingest): tableau - add support for column level lineage (#6264) 2022-10-26 19:42:51 -07:00
Mayuri Nehate
dc1e4d70e4
refactor(ingest): snowflake - update field name for column lineage (#6263) 2022-10-26 16:27:27 -07:00
abelstam12
cc88c8780c
fix(ingest): bigquery - only register failure on exception (#6242) 2022-10-26 16:26:09 -07:00
Tamas Nemeth
94fae0a464
feat(ingest): bigquery - Promoting bigquery-beta to bigquery source (#6222)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 11:15:44 -07:00
Shirshanka Das
46453e4b10
docs(ingest): lookml - add docs for multi-project lookml ingestion (#6284) 2022-10-25 19:24:46 -07:00
Shirshanka Das
abf1b11a5e
fix(quickstart): support for user-level sockets, arch option for forcing quickstart architecture (#6279) 2022-10-25 11:45:27 -07:00
Anirudh181001
aeed817783
feat(ingest): allow for deleting aspects in cli with -p tag (#6190)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:32:21 -07:00
AdamDz
10926ff946
fix(ingestion): passing disable_ssl_verification param value to DatahubRestEmitter (#6241)
Co-authored-by: Adam Dziendziel <adam.dziendziel@grandparade.co.uk>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:49 -07:00
BZ
64c28a2316
feat(ingest/kakfa-connect): support generic connectors (#6173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:31 -07:00
Shirshanka Das
c5fc2ea798
fix(ingest): looker - deps, column level lineage fixes (#6271) 2022-10-24 08:31:48 +02:00
Shirshanka Das
26b4a9eec4
fix(ingest):lookml - better column-level lineage, hive urn generation… (#6254) 2022-10-20 16:39:11 -07:00
Tamas Nemeth
67570763dd
feat(ingest): hive-on-presto - Add option to properly filter hive database (#6247) 2022-10-20 23:03:28 +02:00
Tamas Nemeth
333598fda2
fix(ingest): stateful-ingestion - keep dataset urn case in checkpoints (#6244) 2022-10-19 23:03:21 +02:00
Tamas Nemeth
0545f3cb38
fix(ingest): presto-on-hive - Adding catalog name to the presto on hive urn (#6024)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-19 09:28:33 -07:00
Tamas Nemeth
e54f376f0a
feat(ingest): bigquery-beta - Add option to lowercase urns (#6240) 2022-10-19 14:44:28 +02:00
Mayuri Nehate
6373c43bfe
fix(ingest): snowflake - remove back-ticks from table name when creating urn (#6236) 2022-10-19 12:32:54 +02:00
Tamas Nemeth
9015a43f25
fix(ingest): bigquery-beta - Adding python 3.8 fix for memory footprint util (#6228) 2022-10-18 17:59:31 -07:00
Tamas Nemeth
85268d6845
fix(ingest): bigquery-beta - Add stacktrace for bigquery schema ingest logs (#6226) 2022-10-18 14:42:46 +02:00
Mayuri Nehate
98ba36c5a0
feat(ingest): snowflake - support deletion of containers in snowflake stateful ingestion (#6180) 2022-10-17 19:29:24 +02:00
Harshal Sheth
73fd35888b
build(ingest): remove markupsafe dep and bump pytest-docker (#6201) 2022-10-14 18:59:40 -07:00
Mayuri Nehate
afac3a9ce6
feat(ingest): s3 - allow selfsigned certificate (#6179)
Fixes #5307
2022-10-14 18:54:50 -07:00