1646 Commits

Author SHA1 Message Date
sid-acryl
ad2df22d78
feat(ingestion/dbt): multiple node owner separated by comma (#9740)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-01-31 16:38:26 +01:00
Aseem Bansal
3f9490d901
feat(cli): option to init via username password, add lint for smoke-test (#9675)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-31 14:42:40 +05:30
Ellie O'Neil
d1604b39cc
feat(ingestion/tableau) Ingest tags from datasource (#9730)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-29 17:17:07 -08:00
Harshal Sheth
1498c36875
chore(cli): drop support for python 3.7 (#9731) 2024-01-29 10:50:47 -08:00
Tamas Nemeth
90c88082b1
fix(ingest/s3): Not sorting schema fields to keep original order (#9349) 2024-01-29 14:14:34 +01:00
tom
f7f0b14f37
fix(ingestion/metabase): Fetch Dashboards through Collections (#9631)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 17:51:41 -08:00
Shubham Jagtap
f83a2fab44
fix(ingestion/bigquery): Table-view-snapshot Lineage Bug fix (#9579)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 18:48:41 +05:30
Andrew Sikowitz
53c7790f9a
feat(ingest/metabase): Use new sql parser; reduce error reporting levels (#9714) 2024-01-25 10:36:59 +01:00
Harshal Sheth
9b051e38d6
feat(ingest/dbt): support aws config without region (#9650)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-01-24 14:29:41 -08:00
Aseem Bansal
c4dec931a3
feat(ingest): handling for const in json schema (#9694)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-24 06:44:14 +00:00
Harshal Sheth
19b76c3307
feat(ingest/sql-parser): test redshift temp tables (#9653) 2024-01-23 08:37:53 +01:00
Harshal Sheth
f14dc4ded2
feat(ingest): improve sql tests + dbt reporting (#9649) 2024-01-23 08:37:23 +01:00
sleeperdeep
720296ea46
fix(ingestion/mssql): use platform_instance for mssql dataflow/datajob (#9527)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: sleeperdeep <--global>
2024-01-22 20:25:01 -08:00
Harshal Sheth
b94d463fe0
feat(ingest/lookml): support complex lookml manifests (#9688) 2024-01-22 16:23:56 -08:00
david-leifker
943bb57cbc
feat(backend): structured properties and forms (#9626)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2024-01-22 11:46:04 -06:00
Mayuri Nehate
77df9ec926
feat(ingest/databricks): view upstream lineage for hive metastore (#9657) 2024-01-22 18:07:43 +05:30
Quentin FLEURENT NAMBOT
20ad83d0b1
feat(superset): add some custom properties for dashboards (#9670) 2024-01-20 14:54:10 +05:30
Mayuri Nehate
f2e78db92e
feat(ingest/databricks): ingest hive metastore by default, more docs (#9601)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-18 14:26:33 -08:00
ethan-cartwright
dfb2f7e596
feat(classifier): Add support for excluding list of exact column names (#9472)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
2024-01-16 19:12:23 -08:00
Tamas Nemeth
757eeecc14
feat(ingest/dbt-cloud): Add way to customize dbt cloud access endpoint (#9633)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2024-01-16 16:59:56 -08:00
Harshal Sheth
98e3da42f5
feat(ingest/looker): add backpressure-aware executor (#9615)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-01-12 18:54:08 +01:00
Harshal Sheth
b926a3ede6
feat(ingest/looker): add retries to API calls (#9606) 2024-01-12 13:42:52 +01:00
Mayuri Nehate
b0426ae94b
fix(ingest/nifi): make s3 dataset urn consistent with s3 source (#9599) 2024-01-12 11:07:53 +05:30
Harshal Sheth
f05056aed7
feat(ingest): key-partitioning for rest emitter (#9613) 2024-01-11 17:57:48 -05:00
Harshal Sheth
990bb65fec
feat(ingest/tableau): include platform_instance in browse v1 path (#9607) 2024-01-11 11:32:00 +05:30
Shubham Jagtap
0486319bc8
feat(ingestion/fivetran): Add fivetran bigquery destination support (#9531)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-10 14:18:36 -05:00
Shubham Jagtap
3f9b90158f
fix(ingestion/starburst-trino): Fix validation errors in TrinoJoinedAccessEvent (#9580)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-10 12:56:04 -05:00
Shubham Jagtap
cb8002413e
feat(ingestion/transformer): Add dataset dataproduct transformer (#9491)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-05 17:03:22 -05:00
RyanHolstien
822d0eb014
feat(patch): add dashboardInfo and chartInfo support for patch (#9536) 2024-01-03 15:11:07 -06:00
Harshal Sheth
c9613043c8
fix(ingest): improve kafka-connect test stability (#9519) 2024-01-03 11:28:22 -08:00
Mayuri Nehate
29f2142a2c
feat(databricks): add hive metastore analyze profiling (#9511) 2024-01-02 23:18:43 -05:00
Aseem Bansal
06bd9b988d
fix(cli): upsert for data product external url (#9534) 2023-12-29 21:34:06 +05:30
Diego Reiriz Cores
60347d6735
fix(ingest/mongodb): support disabling schemaSamplingSize (#9295)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-28 16:39:10 +05:30
Shubham Jagtap
3635c1c221
feat(ingestion/bigquery): Use sqlglot_lineage for usage and add more perf timers (#9247)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-12-28 04:54:26 -05:00
Harshal Sheth
0d8568e087
fix(ingest): update dbt type inference (#9512) 2023-12-22 14:59:14 -05:00
Harshal Sheth
52687f3eea
feat(ingest): support CLL for redshift materialized views with auto refresh (#9508) 2023-12-22 08:18:22 +01:00
Harshal Sheth
a18c72083d
feat(ingest): handle multiline string coercion (#9484) 2023-12-21 13:50:39 -05:00
Harshal Sheth
b80d2f471c
fix(ingest/fivetran): only materialize upstream lineage (#9490) 2023-12-21 13:35:34 -05:00
Harshal Sheth
ecda3e6187
feat(ingest): pydantic v2 compatibility (#9434) 2023-12-18 18:26:33 -05:00
Tamas Nemeth
03590a1948
fix(ingest/snowflake) - Fixing snowflake url with default region (#9443) 2023-12-18 13:54:31 -05:00
Harshal Sheth
9d386fbd6f
feat(ingest): enable CLL for dbt by default (#9466) 2023-12-18 11:38:16 +01:00
Harshal Sheth
caef6771b8
feat(ingest/redshift): drop repeated operations (#9440) 2023-12-15 15:07:56 -05:00
Tamas Nemeth
0ea6145a9d
fix(ingest/profiling): Add option to enable external table profiling (#9463) 2023-12-15 00:12:45 +01:00
Shubham Jagtap
1741c07d76
feat(ingestion): Add test_connection methods for important sources (#9334) 2023-12-14 12:31:51 -05:00
Shirshanka Das
ecef50f8fc
fix(ingest): make user_urn and group_urn generation consider user and… (#9026)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-12-14 21:33:36 +05:30
siddiquebagwan-gslab
0d6a5e5df2
feat(ingestion/transformer): create tag if not exist (#9076) 2023-12-14 10:36:28 -05:00
Mayuri Nehate
aac1c55a14
feat(ingest/unity): enable hive metastore ingestion (#9416) 2023-12-14 10:35:06 -05:00
Aseem Bansal
b0de1dc0ce
fix(powerbi): add access token refresh (#9405)
Co-authored-by: elish7lapid <elisheva@foundational.io>
Co-authored-by: treff7es <treff7es@gmail.com>
2023-12-14 18:41:50 +05:30
skrydal
a495d652e0
feat(ingestion): Add typeUrn handling to ownership transformers (#9370) 2023-12-13 14:34:20 -05:00
RyanHolstien
8a1122049c
feat(patch): support fine grained lineage patches (#9408)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-11 16:25:43 -06:00