3089 Commits

Author SHA1 Message Date
Harshal Sheth
582fdf95cd
chore(ingest): upgrade to mypy 1.0.0 (#7313) 2023-02-10 13:24:05 -08:00
Tamas Nemeth
793f303a79
fix(ingest/bigquery): Lowering significantly the memory usage of the BigQuery connector (#7315) 2023-02-10 13:12:02 -08:00
Harshal Sheth
55442042ff
feat(cli): improve startup time (#7292) 2023-02-10 21:36:01 +05:30
Aseem Bansal
ca65f9db18
refactor(github): change github reference to git references (#7308) 2023-02-10 21:07:44 +05:30
Tamas Nemeth
97355bc805
fix(ingest/bigquery): Usage rate limiting and lineage exported log fix (#7297) 2023-02-10 20:22:49 +05:30
Shirshanka Das
fc12fad74c
fix(docs): kafka - update docs to indicate protobuf support (#7280)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-02-09 14:40:00 -08:00
Tamas Nemeth
bf7c00d88d
feat(ingest/bigquery) - Reporting current state of BigQuery ingestion (#7282) 2023-02-08 20:22:49 +01:00
Tamas Nemeth
1402071e48
fix(ingest/bigquery) - Fix for Bigquery parser quoted semicolon in the FROM table name as well (#7277) 2023-02-08 10:18:55 +01:00
Shirshanka Das
e0f9ba1d1f
feat(cli): add 0.10 awareness to upgrade prompt (#7273) 2023-02-07 16:50:40 -08:00
Mayuri Nehate
82e0d10637
feat(ingest/s3): use latest file to infer schema metadata (#7202)
Co-authored-by: John Joyce <john@acryl.io>
2023-02-06 13:40:36 -08:00
Harshal Sheth
b24f206e04
fix(ingest/tableau): handle missing query in tableau views (#7186) 2023-02-06 13:39:26 -08:00
Mayuri Nehate
389aa8c1d0
fix(cli): update message when pulling docker images (#7241)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-02-06 16:35:45 +01:00
Mayuri Nehate
e8c141269b
docs(ingest): update kafka connect doc, simplify starter recipe (#7243)
Co-authored-by: John Joyce <john@acryl.io>
2023-02-06 16:33:41 +01:00
Harshal Sheth
d2bcdc301a
feat(cli): improve docker quickstart (#7184) 2023-02-03 17:46:03 +05:30
Aditya Radhakrishnan
bd17dde7b0
fix(ingest-idp): emit empty GroupMembership when there are no groups (#7196)
Co-authored-by: John Joyce <john@acryl.io>
2023-02-02 11:56:40 -08:00
Teppo Naakka
ba7b9d068e
fix(ingest): redshift # build late binding view lineage when sql written in upper case (#7223) 2023-02-02 20:53:30 +01:00
Harshal Sheth
2cfd82203a
fix(ingest/tableau): implement workbook_page_size parameter (#7216)
Co-authored-by: John Joyce <john@acryl.io>
2023-02-02 11:43:16 -08:00
Harshal Sheth
23730e0e9c
fix(cli/delete): skip references prompt if deleting an aspect (#7220) 2023-02-02 11:42:27 -08:00
Daniel Messias
0d67e188ef
feat(glue): Use table name as human-readable name for Glue ingestion (#7213)
Co-authored-by: John Joyce <john@acryl.io>
2023-02-02 18:04:35 +01:00
Dmitry Bryazgin
1a1f2906db
feat(ingest): Add custom properties to the ldap ingestion (#7125)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-02-01 16:37:14 -08:00
Iaroslav Frolikov
a2e0c60eeb
fix(ingest/bigquery): quoting for APPROX_COUNT_DISTINCT in BigQuery (#7207) 2023-02-01 22:06:17 +01:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements (#6894) 2023-01-31 18:44:37 -06:00
Tamas Nemeth
0697fbcf81
fix(ingest/vertica): Fixing missing container properties (#7197) 2023-01-31 19:52:55 +01:00
Harshal Sheth
ed360e95c4
fix(ingest/tableau): show warning about token expiry for PATs (#7187) 2023-01-31 10:49:22 -08:00
Patrick Franco Braz
8ee9fa1930
feat(ingest): bigquery - extracts lineage metadata from catalog api (#7137) 2023-01-31 15:02:30 +01:00
Harshal Sheth
670342ab64
feat(ingest): switch dbt to use auto_stale_entity_removal (#7160) 2023-01-30 19:59:20 -08:00
Mayuri Nehate
efaf092f5b
fix(ingest/GX): fix snowflake urn generated from connection string (#7173)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-30 16:23:42 -08:00
Peter Szalai
8a2230aea4
fix(quickstart): adding back kafka-setup (#7181)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-01-31 00:34:36 +01:00
Harshal Sheth
b4114c6499
test(ingest/delta-lake): fix spurious directory creation (#7179) 2023-01-30 13:12:18 -08:00
Harshal Sheth
7ace79c153
fix(ingest): switch various sources to auto_stale_entity_removal helper (#7158) 2023-01-30 11:45:12 -08:00
Aseem Bansal
7314aebf71
fix(ingest): azuread group mapping do not stop ingestion (#7169) 2023-01-30 22:27:51 +05:30
mohdsiddique
54562cd2d5
fix(ingestion/powerbi): Not all arguments converted to string (#7157)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-27 18:49:27 +01:00
Harshal Sheth
8c42a1fc2f
fix(ingest/snowflake): fix regression in approx count distinct (#7146) 2023-01-26 16:15:22 -08:00
Harshal Sheth
927d45dda9
feat(ingest): add --log-file option and show CLI logs in UI report (#7118)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-26 09:25:02 -08:00
Harshal Sheth
ca5c6c3811
fix(ingest): use snowflake median function in profiling (#6987) 2023-01-25 23:56:43 -08:00
Aseem Bansal
91c82fa5db
refactor(cli): extract method, change wording (#7134)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-25 10:54:38 -08:00
Tamas Nemeth
b84e2ac5b2
fix(doc/biquery): Updating bigquery capability doc (#7136) 2023-01-25 17:19:04 +01:00
Shirshanka Das
e842164465
feat(datahub-lite): simplify get response (#7131) 2023-01-25 15:30:55 +01:00
Teppo Naakka
2b43bce3d6
feat(ingest): powerbi # add powerbi workspaces to containers (#6532)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-24 15:06:50 -08:00
Pedro Silva
bef59b0928
feat(quickstart): Remove kafka-setup as a hard deployment requirement (#7073) 2023-01-24 16:12:57 +00:00
Aseem Bansal
ab84863757
fix(ci): do not confirm on force for deletion (#7106) 2023-01-24 17:14:09 +05:30
Mert Tunç
6f5cd3f4a9
feat(redshift): Fetch lineage from unload queries (#7041)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-24 12:03:17 +01:00
Harshal Sheth
ceaeb42a2b
feat(ingest): hide cartesian product warnings in GE profiler (#7096) 2023-01-24 08:30:50 +01:00
Tamas Nemeth
e92ce161f9
fix(ingest/profile): properly quoting approx_count_distinct (#7101) 2023-01-23 18:07:14 -08:00
Harshal Sheth
54c5017efd
feat(ingest): move datahub-lite to optional dep and add shim when missing (#7097) 2023-01-20 17:24:43 -08:00
Tamas Nemeth
0cdb5e4b4b
refactor(ingest/containers): Refactoring container creation to common place (#6877) 2023-01-21 00:14:31 +01:00
Harshal Sheth
c5747a4cbd
fix(ingest/snowflake): handle corrupted snowflake OCSP cache file (#7095) 2023-01-20 22:40:36 +01:00
J Feldman
84f7f270ac
fix(profiling): Fix syntax for APPROX_COUNT_DISTINCT on bigquery and snowflake (#7087) 2023-01-20 10:21:58 +01:00
Harshal Sheth
13cc16fbc2
fix(cli/lite): fix datahub lite serve command (#7089) 2023-01-20 10:21:24 +01:00
Harshal Sheth
c4f946c609
fix(ingest/bigquery): clear stateful ingestion correctly (#7075)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-19 18:06:31 +01:00