3975 Commits

Author SHA1 Message Date
Harshal Sheth
81ec296b06
feat(cli): switch default quickstart to v0.10.0 (#7567) 2023-03-13 21:42:05 -07:00
John Joyce
18f387c6ea
fix(cli): Adding exit code to correctly return failure or success (#7520)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-03-13 13:32:40 -07:00
Teppo Naakka
23830423f8
feat(ingest): powerbi # store powerbi entity descriptions (#7154)
Co-authored-by: John Joyce <john@acryl.io>
2023-03-13 13:31:21 -07:00
Harshal Sheth
fbfe43b1cb
feat(ingest): fix edge cases + interface cleanup for file-system APIs (#7533) 2023-03-13 13:14:53 -07:00
Paul Logan
cebc583345
docs(): re-add sources summary page (#7563)
Co-authored-by: Jeff Merrick <jeff@wireform.io>
2023-03-13 11:37:24 -07:00
Shirshanka Das
26cf0a71ab
fix(test): suppress s3 golden file test for specific paths (#7551) 2023-03-12 10:43:02 -07:00
Mayuri Nehate
dac3938077
chore(ingest): snowflake - bump up classification library version to 0.0.6 (#7542) 2023-03-12 10:20:32 -07:00
Shirshanka Das
9a3d3b40ef
feat(openapi): add relationships endpoint (#7547) 2023-03-11 17:25:50 -08:00
Harshal Sheth
759e51bb42
fix(ingest/looker): stringify looker user ids (#7531) 2023-03-10 12:00:37 -08:00
Harshal Sheth
66f255e805
docs(ingest): add guidelines around proactive version pinning (#7534) 2023-03-10 10:16:01 -08:00
Shirshanka Das
a653b8ad30
fix(ingest): delta-lake - support assume aws role (#7524)
Co-authored-by: Dhanya Jayachandra <dhanyajayachandra@MacBook-Pro.local>
Co-authored-by: John Joyce <john@acryl.io>
2023-03-09 20:33:37 -08:00
Harshal Sheth
b82afa89f1
feat(ingest): enable joins across FileBackedDicts + add FileBackedList (#7506) 2023-03-09 15:22:03 -08:00
Harshal Sheth
91a8e5ea16
feat(ingest/powerbi): support PowerBI parameter references (#7523) 2023-03-09 11:14:55 -08:00
J Feldman
aa4228734c
feat(ingest/looker): upgrade to Looker API from 3.1 to 4.0 (#7411)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-03-07 16:49:37 -08:00
Mayuri Nehate
dc2a7d8a46
chore(ingest): remove unused dependency for bigquery (#7510)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 16:43:14 -08:00
NotYuki
bb9835cf21
feat(ingest/vertica): improve vertica type mappings (#7459)
Co-authored-by: Maxim Losnikov <maxim.losnikov@truv.com>
2023-03-07 15:17:38 -08:00
cburroughs
86264b19cf
fix(ingest): limit typing_extensions classes to those available in min version (#7490) 2023-03-07 15:12:20 -08:00
Harshal Sheth
630e80c7e1
fix(ingest): make quickstart error handling more robust (#7513) 2023-03-07 14:40:31 -08:00
Mayuri Nehate
df2c62fa84
fix(ingest/bigquery): fix missing materialized views (#7511)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 14:40:08 -08:00
Arun Vasudevan
e93a17e917
feat(ingest/postgres): support estimated row counts in profiling (#7476)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 11:08:09 -08:00
Peter Szalai
7207c348b8
feat(cli): finalizing quickstart config commit hash (#7509) 2023-03-07 10:47:45 -08:00
cburroughs
cc0772f8d8
feat(ingest): unbundle airflow plugin emitter dependencies (#7493)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 09:07:42 -08:00
mohdsiddique
de719663ff
feat(ingestion): powerbi # support Google BigQuery table lineage (#7502)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 09:07:00 -08:00
Peter Szalai
1d33392761
feat(cli): introduce remote config for quickstart (#7424)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 13:14:24 +01:00
Mayuri Nehate
406b11a9ed
feat(ingest/GX): add urn lowercasing option for GX assertions (#7472) 2023-03-06 20:42:23 -08:00
Harshal Sheth
01ee351c4c
fix(ingest): prevent logging from blowing up on TypeErrors (#7497) 2023-03-03 14:36:55 -08:00
Harshal Sheth
195196ddf1
fix(ingest): redact auth info in curl commands (#7496) 2023-03-03 14:35:29 -08:00
Harshal Sheth
795d76b56c
fix(ingest/tableau): load project workbook hierarchy correctly (#7483) 2023-03-03 11:54:42 -08:00
Harshal Sheth
b4dd1d7d82
chore(ingest): pin acryl-datahub-classify (#7485) 2023-03-03 11:36:20 -05:00
Andrew Sikowitz
7a71b84296
refactor(ingest): Convert FileBackedDict to dataclass for cleaner init (#7469)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-02 19:53:05 -08:00
mohdsiddique
29d171106b
feat(ingest/tableau): project path and container support (#7426)
Co-authored-by: mayurinehate <mayuri.nehate@gslab.com>
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-03-02 16:53:19 -08:00
Kevin G
622688916c
fix(ingest/dbt): check for nodes key before accessing (#7462)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-02 11:40:13 -08:00
Aseem Bansal
1adbc2cab0
chore(ci): upgrade GE version (#7290) 2023-03-02 10:47:38 -08:00
Tamas Nemeth
3f88fb7d16
feat(ingest/bigquery) - Capture dataset labels in bigquery (#7460) 2023-03-02 11:41:19 +01:00
Harshal Sheth
49029943f9
fix(ingest): remove extraneous platform configs (#7454) 2023-03-02 01:10:35 -08:00
Thomas Memenga
18dd7298ad
fix(ingest/s3): propagate s3 endpoint to profiling (#7431) 2023-03-02 01:05:17 -08:00
Tony Ouyang
4f651b0d3d
fix(ingest/bigquery): update bigquery platform_instance capability (#7467) 2023-03-02 00:52:40 -08:00
Harshal Sheth
c648f7376a
refactor(ingest): use auto_stale_entity_removal in json schema source (#7465) 2023-03-02 08:25:41 +01:00
Harshal Sheth
619fad0ae1
fix(ingest/dbt): remove deprecated backcompat_skip_source_on_lineage_edge option (#7466) 2023-03-02 08:24:50 +01:00
Andrew Sikowitz
8101f0d47a
feat(ingest): Introduce FileBackedDict for offloading data to disk (#7461)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Also includes minor refactoring to the bigquery connector
2023-03-01 19:09:51 -05:00
Harshal Sheth
45feb01e3b
fix(ingest/bigquery): simplify type annotations for bigquery usage (#7457)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-01 08:38:36 -08:00
Harshal Sheth
2c3e3c203f
docs(ingest): add details about backwards compatibility guarantees (#7439) 2023-02-28 13:33:58 -08:00
Shirshanka Das
17e85979dd
refactor(ingest): subtypes - standardize (#7437) 2023-02-28 13:11:07 -08:00
Harshal Sheth
376fffeebc
docs(ingest): add more guidelines for writing sources (#7451) 2023-02-28 11:53:43 -08:00
Harshal Sheth
73493c577b
refactor(ingest): avoid allowing extras for all DataHubGraphConfig (#7448) 2023-02-28 10:42:31 -08:00
Harshal Sheth
639bbcfa86
chore(ingest/glue): cleanup deprecated underlying_platform config (#7449) 2023-02-28 10:41:54 -08:00
nachiket-juneja
e07cd2090b
Feat/s3 ingestion enhancement to update schema from latest partition (#7410)
Co-authored-by: Prashant Singh Thakur <prashant.thakur@nucleusteq.com>
2023-02-28 08:58:28 +01:00
Harshal Sheth
3b8b5e8aa4
chore(ingest): cleanup unused files/vars in tests (#7450) 2023-02-28 08:07:34 +01:00
Tamas Nemeth
62e33e03a3
fix(ingest/unity): Use assigned metastore if not metastore listed in unity catalog (#7446) 2023-02-28 08:06:28 +01:00
Tamas Nemeth
77d072b522
fix(ingest/athena): Fix athena source if dbname is not specified in the connection string (#7417) 2023-02-27 22:15:29 +01:00