3975 Commits

Author SHA1 Message Date
siladitya
43ac405c57
fix(metadata-ingestion)glue connector failure when Optional field Type of PartitionKey is absent for a Table (#10052) 2024-03-20 11:02:28 +01:00
Ellie O'Neil
446fbe5784
fix(ingest/slack): add rate limiting to more places in slack source (#10082) 2024-03-20 13:56:06 +05:30
Ellie O'Neil
cc0552ac80
Add owners, tags, glossary terms to dataset yaml loader (#9859) 2024-03-19 17:27:04 -07:00
skrydal
9aa099f4b0
fix(ingestion): Handle Redshift string length limit in Serverless mode (#10051) 2024-03-19 18:54:41 +01:00
Ellie O'Neil
59a26a6543
Update domains docs to include nested domains (#9890)
Co-authored-by: Hyejin Yoon <0327jane@gmail.com>
2024-03-19 16:17:10 +00:00
Harshal Sheth
1f9ce9d7d8
feat(ingest/dbt): point dbt assertions at dbt nodes (#10055) 2024-03-18 18:13:01 -07:00
Harshal Sheth
11eda4bc84
feat(ingest): refactor LineageConfig class (#10074) 2024-03-18 17:34:53 -07:00
Harshal Sheth
f4d4e791b0
feat(ingest): add sql formatter utility (#10064) 2024-03-18 17:28:32 -07:00
Harshal Sheth
630f2a13f6
feat(ingest): check for private import path usages (#10059) 2024-03-18 14:59:18 -07:00
Gabe Lyons
1248635b5a
docs(sdk): Update API guide with example for Acryl (#10072) 2024-03-18 14:55:58 -07:00
Harshal Sheth
656f94b63c
feat(ingest): better rest emitter error message (#10073) 2024-03-18 14:27:43 -07:00
Harshal Sheth
05930560cc
feat(ingest/s3): set default spark version (#10057) 2024-03-18 14:27:01 -07:00
Harshal Sheth
104e78776d
fix(ingest): only auto-enable stateful ingestion if pipeline name is set (#10075) 2024-03-18 13:59:01 -07:00
dushayntAW
3a4bdef44a
fix(ingest/unity): generate sibling and lineage (#9894) 2024-03-18 20:22:17 +05:30
RyanHolstien
84bdf812da
feat(lineage): add priority to via node (#10034) 2024-03-15 21:05:32 -05:00
Harshal Sheth
d4d175da1a
fix(ingest/redshift): tweak lineage v2 queries (#10045) 2024-03-15 12:16:18 -07:00
Mayuri Nehate
77c72dad01
feat(ingest): add classification to bigquery, redshift (#10031) 2024-03-13 22:45:28 -07:00
Harshal Sheth
b0163c4885
feat(ingest): utilities for query logs (#10036) 2024-03-12 23:20:46 -07:00
Tamas Nemeth
4535f2adfd
fix(ingest/redshift): Temp table lineage fix (#10008)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 14:31:00 -07:00
Harshal Sheth
4155a621a6
feat(ingest/dbt): capture both raw and compiled code (#10026) 2024-03-12 12:54:03 -07:00
Mayuri Nehate
2de0e62ac4
feat(ingest): add classification for sql sources (#10013)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 09:23:20 -07:00
Tamas Nemeth
28f16aabb3
fix(ingest/teradata): small teradata improvements (#9953) 2024-03-12 14:22:17 +01:00
skrydal
2265ae9257
feat(ingestion): Support for Server-less Redshift (#9998)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-03-12 09:32:20 +01:00
Harshal Sheth
5937472998
feat(ingest): add DataHubGraph.emit_all method (#10002) 2024-03-11 16:36:18 -07:00
Harshal Sheth
7e2076e852
feat(ingest): add query formatting to sql aggregator (#10025) 2024-03-11 15:34:44 -07:00
Harshal Sheth
20e15ffc58
feat(ingest): improve custom package metadata (#9985) 2024-03-11 14:14:59 -07:00
Harshal Sheth
92a3ac6f11
fix(ingest): use contextvar for cooperative timeout (#10021) 2024-03-11 14:14:39 -07:00
Harshal Sheth
bcae7acc51
fix(ingest/slack): tweak docs for slack source (#10007) 2024-03-11 14:14:25 -07:00
ethan-cartwright
7540e642a5
fix(ingest/redshift): patch instead of replace redshift custom properties (#9293)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-03-11 11:16:44 -07:00
LePuppy
ae3443bcb7
feat(ingest/superset): map awsathena platform name to athena (#10005) 2024-03-11 15:23:42 +01:00
Harshal Sheth
b6956f9a5c
feat(ingest): update sqlglot fork (#10022) 2024-03-11 15:22:30 +01:00
Harshal Sheth
6227754720
feat(ingest/redshift): make query generation configurable (#10000) 2024-03-07 10:19:01 -08:00
dushayntAW
e963047dd0
fix(ingest/unity): creating group urn in case of group (#9951) 2024-03-07 14:44:06 +05:30
Mayuri Nehate
8047d0b514
feat(ingest/snowflake): generate query nodes for snowflake (#9966)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-06 15:11:37 -08:00
dushayntAW
68a26b47b2
fix(ingest/delta-lake): support parsing nested types correctly (#9862) 2024-03-06 19:22:01 +05:30
Harshal Sheth
4fbe8146a4
fix(ingest/redshift): fix stl scan lineage for lineage v2 (#9986) 2024-03-06 09:15:53 +01:00
Harshal Sheth
ab12385cd8
fix(ingest): only import PathSpec when necessary (#9989) 2024-03-05 14:59:07 -08:00
Tamas Nemeth
f597800953
feat(ingestion/cli): Adding check option to validate allow/deny and path_specs (#9983) 2024-03-05 20:29:08 +01:00
Shubham Jagtap
fda5eb89f7
feat(ingest): enable stateful_ingestion by default for DataHub rest sink (#9934) 2024-03-05 11:18:03 -08:00
Harshal Sheth
4189ca9532
feat(ingest/dbt): generate CLL for all node types (#9964) 2024-03-05 10:07:03 -08:00
Shubham Jagtap
4d053f1ad3
fix(ingestion/fivetran): Fix fivetran get connector jobs bug (#9975) 2024-03-05 17:31:28 +01:00
AvaniSiddhapuraAPT
782d33db99
fix(ingest/bigquery): escape special characters for table descriptions (#9932) 2024-03-05 21:00:09 +05:30
Harshal Sheth
5bee25fa20
feat(ingest): add Looks subtype + stop reemitting browsePathV2 (#9978) 2024-03-05 14:27:41 +01:00
Hyejin Yoon
67406aa218
docs: archive old version before 0.12.0 & fix broken links (#9957)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-05 16:05:26 +09:00
Harshal Sheth
9dd0c37830
feat(ingest): fix fspath lint error (#9976) 2024-03-04 17:09:22 -08:00
Harshal Sheth
d987707cde
feat(ingest): speed up to_obj() and validate() (#9969) 2024-03-04 13:31:39 +01:00
Harshal Sheth
70adf73081
fix(ingest/redshift): fix bug in lineage v2 table renames (#9967) 2024-03-04 13:31:11 +01:00
AvaniSiddhapuraAPT
1b4f31bcfe
fix(ingest/json-schema): adding support descriptions for array (#9757) 2024-03-01 13:03:47 -08:00
Harshal Sheth
4c83f10cdb
feat(ingest): throw codegen error on duplicate class names (#9960) 2024-02-29 16:13:43 -08:00
Harshal Sheth
0c7ca4cd3e
docs(ingest/lookml): update known discrepancy list (#9941)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-02-29 12:39:32 -08:00