3975 Commits

Author SHA1 Message Date
Harshal Sheth
990bb65fec
feat(ingest/tableau): include platform_instance in browse v1 path (#9607) 2024-01-11 11:32:00 +05:30
Gerrit-K
8cb6d8845b
fix(ingest): support database_pattern in clickhouse and clickhouse-usage (#9482)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-10 20:24:44 -05:00
Shubham Jagtap
0486319bc8
feat(ingestion/fivetran): Add fivetran bigquery destination support (#9531)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-10 14:18:36 -05:00
Mayuri Nehate
af866eaf95
feat(ingest/snowflake): improve accuracy of computed sample-based pro… (#9600) 2024-01-10 12:59:09 -05:00
Shubham Jagtap
3f9b90158f
fix(ingestion/starburst-trino): Fix validation errors in TrinoJoinedAccessEvent (#9580)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-10 12:56:04 -05:00
Harshal Sheth
39e88ef16d
fix(ingest/bigquery): support google-cloud-bigquery 3.15.0 (#9595) 2024-01-10 09:56:29 +01:00
Harshal Sheth
cca1e9dd49
fix(ingest/redshift): include table type in custom properties (#9576) 2024-01-09 22:08:31 +01:00
Harshal Sheth
e3f2c52d7e
fix(ingest/snowflake): set platform instance for foreign keys (#9577) 2024-01-09 22:07:49 +01:00
Harshal Sheth
aa28837eed
docs(slack): add note about private Slack channels (#9589) 2024-01-09 22:07:14 +01:00
Shubham Jagtap
cb8002413e
feat(ingestion/transformer): Add dataset dataproduct transformer (#9491)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-05 17:03:22 -05:00
Harshal Sheth
031e6fbb6d
fix(ci): ignore binaries from test results upload (#9560) 2024-01-05 12:18:52 +05:30
Harshal Sheth
2268c0c5b6
feat(ingest): track thread count in ingestion report (#9566) 2024-01-04 16:36:09 -05:00
Harshal Sheth
20b7dd3642
fix(ingest/looker): limit dashboard object lifetime to thread (#9564) 2024-01-04 16:34:08 -05:00
RyanHolstien
822d0eb014
feat(patch): add dashboardInfo and chartInfo support for patch (#9536) 2024-01-03 15:11:07 -06:00
Harshal Sheth
f06b5c7820
feat(ingest): improve config loading helpers (#9477) 2024-01-03 15:30:11 -05:00
Harshal Sheth
83b904e379
fix(ingest/looker): add user stats to report (#9505) 2024-01-03 11:28:32 -08:00
Harshal Sheth
c9613043c8
fix(ingest): improve kafka-connect test stability (#9519) 2024-01-03 11:28:22 -08:00
Mayuri Nehate
29f2142a2c
feat(databricks): add hive metastore analyze profiling (#9511) 2024-01-02 23:18:43 -05:00
Aseem Bansal
06bd9b988d
fix(cli): upsert for data product external url (#9534) 2023-12-29 21:34:06 +05:30
Mayuri Nehate
e343b69ce4
fix(ingest/snowflake): explicit set schema if public schema is absent (#9526) 2023-12-28 16:50:13 +05:30
Tony Ouyang
2cd38a469d
fix(ingest): Fix mongodb ingestion when platform_instance is missing from recipe (#9486)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-28 16:39:30 +05:30
Diego Reiriz Cores
60347d6735
fix(ingest/mongodb): support disabling schemaSamplingSize (#9295)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-28 16:39:10 +05:30
Shubham Jagtap
3635c1c221
feat(ingestion/bigquery): Use sqlglot_lineage for usage and add more perf timers (#9247)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-12-28 04:54:26 -05:00
cburroughs
754d881447
build(ingest/feast): upgrade to latest feast version (#9439) 2023-12-27 19:33:41 -05:00
Fernando Marino`
b7a0bbcb3d
feat(ingest/openapi): support proxies and alternate auth schemes (#9492)
Co-authored-by: Fernando Marino <f.marino@rheagroup.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-27 19:24:25 -05:00
Ryan Despain
c804b3c1aa
fix(ingest/fivetran): handle missing user_id for a connection (#9460)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-27 14:49:16 -05:00
Dmytro Kulyk
19f104a109
fix(ingest/glue): glue table partition (Athena v3) profiling parameters retrieval error (#9521)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-27 14:43:28 -05:00
Harshal Sheth
ed5bdfc5ae
feat(ingest/redshift): merge CLL instead of overwriting (#9513) 2023-12-22 23:12:31 +01:00
Harshal Sheth
0d8568e087
fix(ingest): update dbt type inference (#9512) 2023-12-22 14:59:14 -05:00
Harshal Sheth
52687f3eea
feat(ingest): support CLL for redshift materialized views with auto refresh (#9508) 2023-12-22 08:18:22 +01:00
Harshal Sheth
be329986ab
feat(ingest): rename custom package path from models to metadata (#9502) 2023-12-21 20:30:36 -05:00
Harshal Sheth
ca518d6c78
feat(ingest): remove librdkafka hacks (#9507) 2023-12-21 20:28:45 -05:00
Tamas Nemeth
cfc641f0d0
fix(ingest/databricks): Pinning databricks sdk to not fail on mypy issues (#9500) 2023-12-21 20:32:51 +01:00
Harshal Sheth
a18c72083d
feat(ingest): handle multiline string coercion (#9484) 2023-12-21 13:50:39 -05:00
Harshal Sheth
b80d2f471c
fix(ingest/fivetran): only materialize upstream lineage (#9490) 2023-12-21 13:35:34 -05:00
Harshal Sheth
bf813d1d24
fix(ingest): update ingest_stats event with transformer types (#9487) 2023-12-20 15:49:03 -05:00
david-leifker
3777730d78
fix(quickstart): force strings for mysql version (#9485) 2023-12-19 15:30:47 -06:00
Tamas Nemeth
94a1603676
fix(ingest/redshift: Fixing operation query to not return duplicate operations (#9481) 2023-12-19 10:30:21 -05:00
Harshal Sheth
ecda3e6187
feat(ingest): pydantic v2 compatibility (#9434) 2023-12-18 18:26:33 -05:00
Tamas Nemeth
03590a1948
fix(ingest/snowflake) - Fixing snowflake url with default region (#9443) 2023-12-18 13:54:31 -05:00
Harshal Sheth
9d386fbd6f
feat(ingest): enable CLL for dbt by default (#9466) 2023-12-18 11:38:16 +01:00
Harshal Sheth
caef6771b8
feat(ingest/redshift): drop repeated operations (#9440) 2023-12-15 15:07:56 -05:00
Tamas Nemeth
0ea6145a9d
fix(ingest/profiling): Add option to enable external table profiling (#9463) 2023-12-15 00:12:45 +01:00
Shubham Jagtap
1741c07d76
feat(ingestion): Add test_connection methods for important sources (#9334) 2023-12-14 12:31:51 -05:00
Shirshanka Das
ecef50f8fc
fix(ingest): make user_urn and group_urn generation consider user and… (#9026)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-12-14 21:33:36 +05:30
siddiquebagwan-gslab
0d6a5e5df2
feat(ingestion/transformer): create tag if not exist (#9076) 2023-12-14 10:36:28 -05:00
Mayuri Nehate
aac1c55a14
feat(ingest/unity): enable hive metastore ingestion (#9416) 2023-12-14 10:35:06 -05:00
Aseem Bansal
b0de1dc0ce
fix(powerbi): add access token refresh (#9405)
Co-authored-by: elish7lapid <elisheva@foundational.io>
Co-authored-by: treff7es <treff7es@gmail.com>
2023-12-14 18:41:50 +05:30
Seokyun Ha
70e64e8078
feat(ingest): add ingest --no-progress option (#9300) 2023-12-14 14:32:37 +05:30
Arun Vasudevan
32d237b56f
fix(ingest): reduce GraphQL Logs to warning for circuit breaker (#9436) 2023-12-13 23:02:21 +01:00