3975 Commits

Author SHA1 Message Date
Mayuri Nehate
29b99052d7
feat(ingest): tableau - add support for column level lineage (#6264) 2022-10-26 19:42:51 -07:00
Mayuri Nehate
dc1e4d70e4
refactor(ingest): snowflake - update field name for column lineage (#6263) 2022-10-26 16:27:27 -07:00
abelstam12
cc88c8780c
fix(ingest): bigquery - only register failure on exception (#6242) 2022-10-26 16:26:09 -07:00
Tamas Nemeth
94fae0a464
feat(ingest): bigquery - Promoting bigquery-beta to bigquery source (#6222)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 11:15:44 -07:00
Shirshanka Das
46453e4b10
docs(ingest): lookml - add docs for multi-project lookml ingestion (#6284) 2022-10-25 19:24:46 -07:00
Shirshanka Das
abf1b11a5e
fix(quickstart): support for user-level sockets, arch option for forcing quickstart architecture (#6279) 2022-10-25 11:45:27 -07:00
Anirudh181001
aeed817783
feat(ingest): allow for deleting aspects in cli with -p tag (#6190)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:32:21 -07:00
AdamDz
10926ff946
fix(ingestion): passing disable_ssl_verification param value to DatahubRestEmitter (#6241)
Co-authored-by: Adam Dziendziel <adam.dziendziel@grandparade.co.uk>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:49 -07:00
BZ
64c28a2316
feat(ingest/kakfa-connect): support generic connectors (#6173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:31 -07:00
Shirshanka Das
c5fc2ea798
fix(ingest): looker - deps, column level lineage fixes (#6271) 2022-10-24 08:31:48 +02:00
Shirshanka Das
26b4a9eec4
fix(ingest):lookml - better column-level lineage, hive urn generation… (#6254) 2022-10-20 16:39:11 -07:00
Tamas Nemeth
67570763dd
feat(ingest): hive-on-presto - Add option to properly filter hive database (#6247) 2022-10-20 23:03:28 +02:00
Tamas Nemeth
333598fda2
fix(ingest): stateful-ingestion - keep dataset urn case in checkpoints (#6244) 2022-10-19 23:03:21 +02:00
Tamas Nemeth
0545f3cb38
fix(ingest): presto-on-hive - Adding catalog name to the presto on hive urn (#6024)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-19 09:28:33 -07:00
Tamas Nemeth
e54f376f0a
feat(ingest): bigquery-beta - Add option to lowercase urns (#6240) 2022-10-19 14:44:28 +02:00
Mayuri Nehate
6373c43bfe
fix(ingest): snowflake - remove back-ticks from table name when creating urn (#6236) 2022-10-19 12:32:54 +02:00
Tamas Nemeth
9015a43f25
fix(ingest): bigquery-beta - Adding python 3.8 fix for memory footprint util (#6228) 2022-10-18 17:59:31 -07:00
Tamas Nemeth
85268d6845
fix(ingest): bigquery-beta - Add stacktrace for bigquery schema ingest logs (#6226) 2022-10-18 14:42:46 +02:00
Mayuri Nehate
98ba36c5a0
feat(ingest): snowflake - support deletion of containers in snowflake stateful ingestion (#6180) 2022-10-17 19:29:24 +02:00
Harshal Sheth
73fd35888b
build(ingest): remove markupsafe dep and bump pytest-docker (#6201) 2022-10-14 18:59:40 -07:00
Mayuri Nehate
afac3a9ce6
feat(ingest): s3 - allow selfsigned certificate (#6179)
Fixes #5307
2022-10-14 18:54:50 -07:00
Mayuri Nehate
5377943d8b
feat(ingest): great-expectations - attach logging handler to write logs to stdout (#6192) 2022-10-14 18:52:12 -07:00
Harshal Sheth
fbe4b7278a
fix(ingest): only restrict GE version for hive (#6170) 2022-10-14 17:17:56 -07:00
Mayuri Nehate
3fc6364979
fix(ingest): catch errors when profiling for sample values (#6194) 2022-10-14 17:15:56 -07:00
Harshal Sheth
43f7b568e3
build: upgrade gradle wrapper (#6203) 2022-10-14 17:04:11 -07:00
Harshal Sheth
1b2b2827df
fix(ingest): use corpGroup instead of corpgroup (#6202) 2022-10-14 17:02:30 -07:00
Harshal Sheth
d08f5f7cdd
feat(ingest): replace base85's pickle with json (#6178) 2022-10-14 14:48:44 -07:00
Tamas Nemeth
5a2bdea77e
docs: dataset usage and query history feature guide (#5900) 2022-10-14 21:43:50 +02:00
Arne Laponin
bbdc185ac2
feat(ingest): add platform instance to tableau (#5978)
Co-authored-by: Arne Laponin <arne.laponin@thoughtworks.com>
Co-authored-by: Amanda Hernando <110099762+amanda-her@users.noreply.github.com>
Co-authored-by: amanda-her <912amandahernando@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-13 18:44:06 -07:00
Harshal Sheth
521e61d3bf
fix(ingest): Closeable as a context manager (#6067) 2022-10-13 12:49:50 -07:00
Harshal Sheth
3f17408fe8
fix(ingest): report warning/error counts correctly (#6128) 2022-10-13 12:49:08 -07:00
Harshal Sheth
82d480dfad
feat(ingest): support self-signed certs in Tableau (#6172) 2022-10-13 12:48:51 -07:00
Harshal Sheth
8030ac83bc
feat(ingest/tableau): emit status aspects + streamline stateful ingestion (#6188) 2022-10-13 12:48:28 -07:00
Harshal Sheth
380d0e86cc
fix(ingest): add lower bound for ujson dep version (#6189) 2022-10-13 12:48:05 -07:00
Harshal Sheth
09616ee2b3
feat(ingest): include instance in container dataPlatform when provided (#6083) 2022-10-13 11:29:54 -07:00
Tamas Nemeth
0162074a37
fix(ingest) - bigquery-beta - Using table ref instead of table id (#6193) 2022-10-13 10:03:33 +02:00
Tamas Nemeth
6e34cd6001
feat(ingest): bigquery-beta - Parsing view ddl definition for lineage (#6187)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-12 18:24:04 -07:00
Harshal Sheth
c7cc0af6ae
feat(ingest/tableau): support dashboard tags (#6185) 2022-10-12 15:34:35 -07:00
Harshal Sheth
e70c0ac4b6
feat(ingest): include raw s3 paths if s3 source (#6168) 2022-10-11 15:55:00 -07:00
Amanda Hernando
ffa838e00e
feat(ingestion): Tableau stateful ingestion (#6094) 2022-10-11 15:52:15 -07:00
Harshal Sheth
95d1e01195
feat(ingest): infer aspect name from type in get_aspect (#6033) 2022-10-11 13:35:41 -07:00
Harshal Sheth
0427122f5d
fix(ingest): allow hiding some fields from the schema (#6077) 2022-10-10 20:46:59 -07:00
Tamas Nemeth
128e3a8970
fix(ingest): bigquery-beta - turning sql parsing off in lineage extraction (#6163)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-10 20:30:29 -07:00
Harshal Sheth
d569734193
refactor(ingest): Tableau cleanup (#6131)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-10 15:52:34 -07:00
Harshal Sheth
879894d702
feat(ingest): airflow - support enabled flag for airflow config (#6089) 2022-10-11 00:42:13 +02:00
Mayuri Nehate
7b88de89d5
fix(ingest): snowflake - allow profiling to work with geography type (#6162) 2022-10-10 08:05:09 -07:00
Harry Nguyen
e9f615442b
fix(ingest): bigquery - Fix syntax error in get_all_schema_tables_query (#6159)
Co-authored-by: Nguyen Tri Hieu <nhieu@fossil.com>
2022-10-10 10:39:30 +02:00
Shirshanka Das
e9c4c823d8
fix(ingest): bigquery-beta - ensure that status aspect is emitted for… (#6154)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-08 16:00:45 -07:00
Harshal Sheth
30014072ad
feat(ingest): lookml - add support for local/remote dependencies (#6150) 2022-10-07 13:14:21 -07:00
Tamas Nemeth
0f04cd6431
fix(ingest): bigquery-beta - eliminate the need for data.read permission for table schema (#6146)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-07 10:19:16 -07:00