3975 Commits

Author SHA1 Message Date
Harshal Sheth
c4f946c609
fix(ingest/bigquery): clear stateful ingestion correctly (#7075)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-19 18:06:31 +01:00
Shirshanka Das
f31ff9c91e
feat(datahub-lite): adding tab completion, small serialization fixes (#7079) 2023-01-19 11:07:07 +01:00
Shirshanka Das
bdcc356cc5
feat(datahub-lite): introduces a new experimental lightweight impleme… (#7052) 2023-01-18 19:18:56 -08:00
Harshal Sheth
e23eb7108f
feat(ingest): reporting revamp, part 1 (#7031) 2023-01-18 13:34:32 -08:00
Harshal Sheth
d7aa61285b
fix(ingest): support git clone of non-github repos (#7065) 2023-01-18 13:30:24 -08:00
Harshal Sheth
35bd73a28b
feat(ingest): fix handling of unions with aliases in post restli conversion (#7058) 2023-01-18 09:29:46 -08:00
Tim
e2ad881d79
refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location (#7044)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-18 09:25:37 -08:00
Harshal Sheth
fc41f455a0
feat(ingest): support snapshots in dbt and dbt-cloud (#7062)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-18 08:35:03 -08:00
Mayuri Nehate
4e7faa5503
fix(ingest/tableau): fix node limit exceeded error for workbooks query (#7068) 2023-01-18 14:34:40 +01:00
Harshal Sheth
afaee58ded
fix(ingest): preserve dbt column name casing (#7063) 2023-01-18 11:57:46 +01:00
Harshal Sheth
bc9adfa441
ci(ingest): add flag to skip tests but run codegen during release (#7067) 2023-01-18 11:55:07 +01:00
Harshal Sheth
f39ed5244c
fix(ingest): support multiple entity_registry sections (#7066) 2023-01-18 15:12:08 +05:30
Remi
a3f4c40422
feat(ingestion): pull metabase database, schema names from raw query and api (#7039) 2023-01-17 20:06:55 -08:00
Teppo Naakka
87b3a5d0fc
feat(ingest): extract powerbi endorsements to tags (#6638) 2023-01-17 19:47:15 -08:00
Harshal Sheth
cb12910b6b
feat(ingest): add entity registry in codegen (#6984)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-17 19:41:43 -08:00
Harshal Sheth
890dae0199
fix(ingest): temporarily disable vertica tests (#7059) 2023-01-17 12:37:16 -08:00
jx2lee
7192f1699d
docs(dbt): fix indentation in dbt meta mapping docs (#7045)
Co-authored-by: jj.lee <jj.lee@coinone.com>
2023-01-17 10:35:04 -08:00
Mayuri Nehate
7607c04ffa
fix(ingest/kafka): fix ResourceType import error for confluent_kafka<1.9.0 (#7046)
Fixes https://github.com/datahub-project/datahub/issues/7020
2023-01-17 10:33:46 -08:00
Rajasekhar-Vuppala
cd9fc26a25
feat(ingest/vertica): Adding Vertica as source in Datahub UI (#7010)
Co-authored-by: Vishal <vishal.k@simplify3x.com>
Co-authored-by: VISHAL KUMAR <110387730+vishalkSimplify@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-13 13:23:32 -08:00
Tamas Nemeth
b238272dda
fix(ingest/bigquery): Turning some usage warning message to debug log as it caused confusion (#7024) 2023-01-13 12:30:52 -08:00
John Joyce
b8d8d198c5
feat(ingest): Ingest Previews for Looker Charts, Dashboards, and Explores (#6941) 2023-01-13 10:25:48 -08:00
mohdsiddique
2ae8fe5868
feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion (#7034)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-13 08:31:47 -08:00
Harshal Sheth
f0432ee101
chore(ingest): remove duplicate data_platform.json file (#7026) 2023-01-12 20:47:08 -08:00
Harshal Sheth
9579e69170
fix(ingest/looker): add clarity in chart input parsing logs (#7003) 2023-01-12 20:40:19 -08:00
Teppo Naakka
ad9a5a1832
fix(ingest): powerbi # use display name field as title for powerbi report page (#7017) 2023-01-12 08:12:30 -08:00
mohdsiddique
dcf389d35f
feat(ingestion): Tableau # Embed links (#6994) 2023-01-11 10:57:48 -08:00
Harshal Sheth
ff49d943bc
fix(ingest): remove dead code from tests (#7005)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-11 10:53:05 -08:00
Harshal Sheth
93dd87a14b
fix(ingest/snowflake): fix type annotations + refactor get_connect_args (#7004) 2023-01-10 18:47:11 -08:00
Aseem Bansal
2fbdd266f8
fix(ingest): bigquery - views in case more than 1 datasets with views (#6995)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-10 11:22:04 -08:00
Aseem Bansal
c82d4fb2fb
fix(docs): build and broken snowflake docs fix (#6997) 2023-01-10 22:52:36 +05:30
서재권(Data Platform)
9578e418c9
fix(ingest): kafka-connect - support newer version of debezium (#6943)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-09 23:13:00 -08:00
Harshal Sheth
fb758f3867
chore(ingest): finish removing feast-legacy (#6985) 2023-01-09 16:06:14 -08:00
Tamas Nemeth
6fdb19067c
fix(ingest): profiling - Fixing issue with the wrong timestamp stored in check (#6978)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-09 15:50:55 -08:00
dependabot[bot]
78e85398fc
chore(deps): bump certifi from 2020.12.5 to 2022.12.7 in /metadata-ingestion/src/datahub/ingestion/source/feast_image (#6979)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-09 13:16:47 -08:00
Harshal Sheth
432feaa16d
feat(ingest): mark database_alias and env as deprecated (#6901) 2023-01-09 19:58:19 +05:30
danielli-ziprecruiter
0ffb353252
feat(ingest/glue): emit s3 lineage for s3a and s3n schemes (#6788)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-09 07:18:36 -05:00
Aseem Bansal
192aac4271
chore: misc fixes (#6966) 2023-01-09 14:12:36 +05:30
Mayuri Nehate
9f20a23e00
fix(ingest): unfreeze bigquery/snowflake column dataclass (#6921) 2023-01-09 09:07:12 +01:00
Paul Logan
f085ec225b
Docs fixes week of 12 22 (#6963)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-06 16:14:49 -08:00
Harshal Sheth
211c30fe30
fix(ingest): add missing dep for powerbi (#6969) 2023-01-06 18:16:32 -05:00
VISHAL KUMAR
96ac4c431f
feat(ingest/vertica): support projections and lineage in vertica (#6785)
Co-authored-by: mraman2512 <MY_mramaan2512@gmail.com>
Co-authored-by: Aman.Kumar <64635307+mraman2512@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-06 16:20:19 -05:00
Lucas Roesler
1088182eb5
feat(ingest/postgres): emit lineage for postgres views (#6953) 2023-01-06 15:53:46 -05:00
Aseem Bansal
d55ad6ca14
fix(ci): restrict GE to fix build issues (#6967) 2023-01-06 18:25:36 +05:30
Harshal Sheth
feaab3b352
fix(ingest/unity): simplify MCP generation and reporting (#6911)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-05 01:47:42 -05:00
Harshal Sheth
f651646d3d
chore(ingest): remove inferred args to MCPW, part 2 (#6905) 2023-01-04 23:29:56 -05:00
Harshal Sheth
8b1dc4bbdf
fix(ingest): use branch info when cloning git repos (#6937) 2023-01-04 16:52:16 -08:00
Harshal Sheth
9bb1c155bd
chore(ingest): partially revert pyspark dep from #6908 (#6954) 2023-01-04 16:51:44 -08:00
Harshal Sheth
e97903f7f6
chore(ingest): unpin pydantic dep (#6909) 2023-01-04 16:31:04 -08:00
Fredrik Sannholm
e0aa812621
feat(ingest): allow extracting snowflake tags (#6500) 2023-01-04 16:05:23 -05:00
Harshal Sheth
6bc85502ba
feat(ingest): add include_table_location_lineage flag for SQL common (#6934) 2023-01-04 14:30:33 -05:00