3975 Commits

Author SHA1 Message Date
Harshal Sheth
0e418b527e
fix(ingest): upgrade pytest-docker (#9765) 2024-02-01 16:33:15 -08:00
sid-acryl
533130408a
feat(ingestion/redshift): collapse lineage to permanent table (#9704)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: treff7es <treff7es@gmail.com>
2024-02-01 12:47:09 -08:00
sid-acryl
874c683f2a
fix(ingest/transformer): generate ownership aspect from handle_end_of_stream (#9720) 2024-01-31 14:25:08 -08:00
sid-acryl
ad2df22d78
feat(ingestion/dbt): multiple node owner separated by comma (#9740)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-01-31 16:38:26 +01:00
Aseem Bansal
3f9490d901
feat(cli): option to init via username password, add lint for smoke-test (#9675)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-31 14:42:40 +05:30
Aseem Bansal
aa98f4897a
feat(cli): add sibling helper CLI command (#9744) 2024-01-30 17:54:09 +05:30
Aseem Bansal
8559ebae20
feat(ingest/okta): option to only ingest users from filtered groups (#9738) 2024-01-30 12:49:07 +05:30
Ellie O'Neil
d1604b39cc
feat(ingestion/tableau) Ingest tags from datasource (#9730)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-29 17:17:07 -08:00
Shirshanka Das
1d06d38b68
feat(platform): add support for via nodes (#9733) 2024-01-29 16:03:27 -08:00
Harshal Sheth
1498c36875
chore(cli): drop support for python 3.7 (#9731) 2024-01-29 10:50:47 -08:00
Tamas Nemeth
5735eb3a55
fix(ingest/test): Fixing breaking change in moto 5.0 library (#9736) 2024-01-29 19:42:30 +05:30
Tamas Nemeth
90c88082b1
fix(ingest/s3): Not sorting schema fields to keep original order (#9349) 2024-01-29 14:14:34 +01:00
Harshal Sheth
2bb4b73f98
fix(ingest/metabase): add missing sql parser dep (#9725) 2024-01-26 23:03:16 +01:00
Harshal Sheth
5adb799f13
fix(cli): fix example data contract yaml + update airflow codecov (#9707) 2024-01-26 23:02:52 +01:00
Dmytro Kulyk
fc27ab2fb5
fix(ingest/glue): Profiling breaks for non-partitioned tables due to absent Table.PartitionKeys (#9591) 2024-01-26 10:54:06 -08:00
tom
f7f0b14f37
fix(ingestion/metabase): Fetch Dashboards through Collections (#9631)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 17:51:41 -08:00
Shubham Jagtap
f83a2fab44
fix(ingestion/bigquery): Table-view-snapshot Lineage Bug fix (#9579)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 18:48:41 +05:30
Andrew Sikowitz
53c7790f9a
feat(ingest/metabase): Use new sql parser; reduce error reporting levels (#9714) 2024-01-25 10:36:59 +01:00
Harshal Sheth
9d8e2b9067
feat(ingest/tableau): map trino_jdbc platform type (#9708) 2024-01-24 14:38:25 -08:00
Harshal Sheth
9b051e38d6
feat(ingest/dbt): support aws config without region (#9650)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-01-24 14:29:41 -08:00
Dimitri
7ae88d97ab
docs(openapi): improve description fields (#9690) 2024-01-24 12:29:28 +05:30
Harshal Sheth
2f36817e95
refactor(ingest): simplify adding aspects to MCEs in transformers (#9686) 2024-01-24 12:28:46 +05:30
Aseem Bansal
c4dec931a3
feat(ingest): handling for const in json schema (#9694)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-24 06:44:14 +00:00
Harshal Sheth
35c4df1e9b
feat(ingest/looker): add browse paths for charts (#9639)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-01-23 08:28:08 -08:00
Harshal Sheth
19b76c3307
feat(ingest/sql-parser): test redshift temp tables (#9653) 2024-01-23 08:37:53 +01:00
Harshal Sheth
f14dc4ded2
feat(ingest): improve sql tests + dbt reporting (#9649) 2024-01-23 08:37:23 +01:00
sleeperdeep
720296ea46
fix(ingestion/mssql): use platform_instance for mssql dataflow/datajob (#9527)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: sleeperdeep <--global>
2024-01-22 20:25:01 -08:00
Harshal Sheth
b94d463fe0
feat(ingest/lookml): support complex lookml manifests (#9688) 2024-01-22 16:23:56 -08:00
david-leifker
943bb57cbc
feat(backend): structured properties and forms (#9626)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2024-01-22 11:46:04 -06:00
Dimitri
1d16e42964
feat(ingest/dynamodb): add domain arg (#9658)
Co-authored-by: Dimitri GRISARD <dgrisard-externe@bedrockstreaming.com>
2024-01-22 19:16:57 +05:30
Mayuri Nehate
77df9ec926
feat(ingest/databricks): view upstream lineage for hive metastore (#9657) 2024-01-22 18:07:43 +05:30
Quentin FLEURENT NAMBOT
344eeaebc7
feat(ingest/superset): add domains and platform_instance support (#9647) 2024-01-21 15:38:44 -05:00
Dimitri
ce65b9cb3f
feat(openapi): allow swagger 3.x (#9646)
Co-authored-by: Dimitri GRISARD <dgrisard-externe@bedrockstreaming.com>
2024-01-20 14:55:40 +05:30
Quentin FLEURENT NAMBOT
6116599458
fix(superset): handle comma in dataset table name (#9656) 2024-01-20 14:54:23 +05:30
Quentin FLEURENT NAMBOT
20ad83d0b1
feat(superset): add some custom properties for dashboards (#9670) 2024-01-20 14:54:10 +05:30
Pedro Silva
0906ce832d
fix(docs): Correct Kafka Connect sink documentation (#9672) 2024-01-19 16:45:22 -08:00
Andrew Sikowitz
f993f50a04
feat(ingest/sql-parsing): Support file backed dict in SqlParsingBuilder for lineage (#9654) 2024-01-18 23:12:20 -05:00
Mayuri Nehate
f2e78db92e
feat(ingest/databricks): ingest hive metastore by default, more docs (#9601)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-18 14:26:33 -08:00
Harshal Sheth
cf200a32ae
fix(docs-website): update copy around the video on index (#9661) 2024-01-18 12:33:19 -08:00
Aseem Bansal
dadd51e750
feat(cli): add threads to sink report (#9642) 2024-01-17 16:57:44 +05:30
ethan-cartwright
dfb2f7e596
feat(classifier): Add support for excluding list of exact column names (#9472)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
2024-01-16 19:12:23 -08:00
cburroughs
acfc315b57
build(ingest/feast): upgrade to latest feast version (0.35.x) (#9627) 2024-01-16 17:28:56 -08:00
Tamas Nemeth
757eeecc14
feat(ingest/dbt-cloud): Add way to customize dbt cloud access endpoint (#9633)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2024-01-16 16:59:56 -08:00
Harshal Sheth
d69ed9a321
feat(ingest/dbt): speed up dbt CLL (#9617) 2024-01-16 10:34:13 +01:00
Tony Ouyang
33e32942b3
fix(ingest/mongodb): Fix downsampling the collection schema output undetermined (#9612) 2024-01-12 13:03:19 -05:00
Harshal Sheth
98e3da42f5
feat(ingest/looker): add backpressure-aware executor (#9615)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-01-12 18:54:08 +01:00
Harshal Sheth
b926a3ede6
feat(ingest/looker): add retries to API calls (#9606) 2024-01-12 13:42:52 +01:00
Mayuri Nehate
b0426ae94b
fix(ingest/nifi): make s3 dataset urn consistent with s3 source (#9599) 2024-01-12 11:07:53 +05:30
Harshal Sheth
f05056aed7
feat(ingest): key-partitioning for rest emitter (#9613) 2024-01-11 17:57:48 -05:00
Harshal Sheth
5476b52d50
feat(ingest/tableau): add retries on OSErrors (#9596) 2024-01-11 19:22:59 +05:30