nachiket-juneja
e07cd2090b
Feat/s3 ingestion enhancement to update schema from latest partition ( #7410 )
...
Co-authored-by: Prashant Singh Thakur <prashant.thakur@nucleusteq.com>
2023-02-28 08:58:28 +01:00
Harshal Sheth
3b8b5e8aa4
chore(ingest): cleanup unused files/vars in tests ( #7450 )
2023-02-28 08:07:34 +01:00
Tamas Nemeth
62e33e03a3
fix(ingest/unity): Use assigned metastore if not metastore listed in unity catalog ( #7446 )
2023-02-28 08:06:28 +01:00
Tamas Nemeth
14a660428e
fix(ingest/bigquery): Querying table metadata details in batch properly ( #7429 )
2023-02-27 11:10:24 +01:00
Shirshanka Das
221b1ae801
fix(ingest): lookml - add support for includes, extends, view_name i… ( #7428 )
2023-02-24 12:05:21 -08:00
Shirshanka Das
95750317e1
refactor(ingest): lookml - fix up golden files in normalized form ( #7423 )
2023-02-24 00:10:18 -08:00
Andrew Sikowitz
0532cc9056
fix(ingest/bigquery) Filter upstream lineage by list of existing tables ( #7415 )
...
Co-authored-by: mayurinehate <mayuri.nehate@gslab.com>
- Creates global stores table_refs and view_upstream_tables when extracting lineage
- Moves lineage processing to the end, after schema processing
- Adds `project_ids` config option to specify multiple projects to ingest; adds corresponding tests
- Changes `created` timestamps to `auditStamp` on `UpstreamClass`; uses VIEW type for lineage identified through view ddl parsing
2023-02-23 19:40:00 -05:00
Tamas Nemeth
4c1bf18f9a
feat(ingest/bigquery) - Emit cross-project usage from gcp logs ( #7364 )
2023-02-22 18:53:35 -05:00
Andrew Sikowitz
e82e284982
fix(ingest/kafka): Remove topic from kafka browse path ( #7398 )
2023-02-22 18:38:08 -05:00
Mayuri Nehate
d436ab9f9b
feat(ingest/kafka-connect): add config to lowercase urns, do not emit… ( #7393 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-02-22 11:42:44 -08:00
Andrew Sikowitz
2764c44977
fix(ingest): Do not require platform_instance for stateful ingestion ( #7397 )
2023-02-21 21:27:44 -05:00
서재권(Data Platform)
3068e7f0b1
fix(ingest/oracle) add database name to oracle urn name ( #7016 )
2023-02-21 13:50:24 -05:00
Andrew Sikowitz
8fd2cc5f20
fix(ingest/snowflake): Improve memory usage of metadata extraction ( #7349 )
2023-02-20 14:46:10 +01:00
Aseem Bansal
986086ae00
test(cli): add check for missing init files ( #7378 )
2023-02-20 18:41:12 +05:30
Shirshanka Das
07e4d0696f
feat(ingest): json-schema - add json schema support for files and kaf… ( #7361 )
2023-02-19 08:43:13 -08:00
Teppo Naakka
702221089d
feat(powerbi): add chart entities to similar browsepath as dashboards ( #7293 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-02-17 13:38:48 -08:00
Aseem Bansal
372f673aef
chore(ci): mark tests correctly ( #7337 )
2023-02-15 16:32:53 +05:30
skrydal
8207e4637a
fix(ingest/tableau): make Tableau ingestor resilient to timeout exceptions ( #7333 )
2023-02-15 11:21:31 +01:00
mohdsiddique
3a095f960f
feat(ingestion): powerbi # Configurable Admin API ( #7055 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-02-14 09:58:34 -08:00
Mayuri Nehate
2cffec9452
fix(check upgrade): update logic to compare server and client version ( #7238 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-02-13 13:09:38 -08:00
Andrew Sikowitz
8901498582
fix(transformers): pattern add domain transformer - enable replace_existing ( #7317 )
2023-02-13 12:52:44 -08:00
Tamas Nemeth
f10d622e47
fix(ingest/bigquery): Improve memory usage of lineage extraction ( #7326 )
2023-02-13 19:59:11 +01:00
Tamas Nemeth
b34e4fe1f1
fix(ingest/bigquery): Fix for table cache was not cleared ( #7323 )
2023-02-13 19:04:19 +01:00
Harshal Sheth
55442042ff
feat(cli): improve startup time ( #7292 )
2023-02-10 21:36:01 +05:30
Aseem Bansal
ca65f9db18
refactor(github): change github reference to git references ( #7308 )
2023-02-10 21:07:44 +05:30
Tamas Nemeth
1402071e48
fix(ingest/bigquery) - Fix for Bigquery parser quoted semicolon in the FROM table name as well ( #7277 )
2023-02-08 10:18:55 +01:00
Aditya Radhakrishnan
bd17dde7b0
fix(ingest-idp): emit empty GroupMembership when there are no groups ( #7196 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-02-02 11:56:40 -08:00
Daniel Messias
0d67e188ef
feat(glue): Use table name as human-readable name for Glue ingestion ( #7213 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-02-02 18:04:35 +01:00
Dmitry Bryazgin
1a1f2906db
feat(ingest): Add custom properties to the ldap ingestion ( #7125 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-02-01 16:37:14 -08:00
Harshal Sheth
db1a0f13f3
fix(ingest): fix issue in glue tests ( #7185 )
2023-01-30 21:51:21 -08:00
Harshal Sheth
670342ab64
feat(ingest): switch dbt to use auto_stale_entity_removal ( #7160 )
2023-01-30 19:59:20 -08:00
mohdsiddique
54562cd2d5
fix(ingestion/powerbi): Not all arguments converted to string ( #7157 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-27 18:49:27 +01:00
Harshal Sheth
927d45dda9
feat(ingest): add --log-file option and show CLI logs in UI report ( #7118 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-26 09:25:02 -08:00
Teppo Naakka
2b43bce3d6
feat(ingest): powerbi # add powerbi workspaces to containers ( #6532 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-01-24 15:06:50 -08:00
Harshal Sheth
45f50d2614
test(ingest): fix kafka admin client mocking ( #7098 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-23 16:22:20 +01:00
Tamas Nemeth
0cdb5e4b4b
refactor(ingest/containers): Refactoring container creation to common place ( #6877 )
2023-01-21 00:14:31 +01:00
Harshal Sheth
e23eb7108f
feat(ingest): reporting revamp, part 1 ( #7031 )
2023-01-18 13:34:32 -08:00
Harshal Sheth
d7aa61285b
fix(ingest): support git clone of non-github repos ( #7065 )
2023-01-18 13:30:24 -08:00
Harshal Sheth
35bd73a28b
feat(ingest): fix handling of unions with aliases in post restli conversion ( #7058 )
2023-01-18 09:29:46 -08:00
Tim
e2ad881d79
refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location ( #7044 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-01-18 09:25:37 -08:00
Harshal Sheth
fc41f455a0
feat(ingest): support snapshots in dbt and dbt-cloud ( #7062 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-18 08:35:03 -08:00
Mayuri Nehate
4e7faa5503
fix(ingest/tableau): fix node limit exceeded error for workbooks query ( #7068 )
2023-01-18 14:34:40 +01:00
Teppo Naakka
87b3a5d0fc
feat(ingest): extract powerbi endorsements to tags ( #6638 )
2023-01-17 19:47:15 -08:00
Harshal Sheth
cb12910b6b
feat(ingest): add entity registry in codegen ( #6984 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-17 19:41:43 -08:00
Harshal Sheth
890dae0199
fix(ingest): temporarily disable vertica tests ( #7059 )
2023-01-17 12:37:16 -08:00
John Joyce
b8d8d198c5
feat(ingest): Ingest Previews for Looker Charts, Dashboards, and Explores ( #6941 )
2023-01-13 10:25:48 -08:00
mohdsiddique
2ae8fe5868
feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion ( #7034 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-13 08:31:47 -08:00
Teppo Naakka
ad9a5a1832
fix(ingest): powerbi # use display name field as title for powerbi report page ( #7017 )
2023-01-12 08:12:30 -08:00
mohdsiddique
dcf389d35f
feat(ingestion): Tableau # Embed links ( #6994 )
2023-01-11 10:57:48 -08:00
Harshal Sheth
ff49d943bc
fix(ingest): remove dead code from tests ( #7005 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-01-11 10:53:05 -08:00
