3089 Commits

Author SHA1 Message Date
Shirshanka Das
f31ff9c91e
feat(datahub-lite): adding tab completion, small serialization fixes (#7079) 2023-01-19 11:07:07 +01:00
Shirshanka Das
bdcc356cc5
feat(datahub-lite): introduces a new experimental lightweight impleme… (#7052) 2023-01-18 19:18:56 -08:00
Harshal Sheth
e23eb7108f
feat(ingest): reporting revamp, part 1 (#7031) 2023-01-18 13:34:32 -08:00
Harshal Sheth
d7aa61285b
fix(ingest): support git clone of non-github repos (#7065) 2023-01-18 13:30:24 -08:00
Harshal Sheth
35bd73a28b
feat(ingest): fix handling of unions with aliases in post restli conversion (#7058) 2023-01-18 09:29:46 -08:00
Tim
e2ad881d79
refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location (#7044)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-18 09:25:37 -08:00
Harshal Sheth
fc41f455a0
feat(ingest): support snapshots in dbt and dbt-cloud (#7062)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-18 08:35:03 -08:00
Mayuri Nehate
4e7faa5503
fix(ingest/tableau): fix node limit exceeded error for workbooks query (#7068) 2023-01-18 14:34:40 +01:00
Harshal Sheth
afaee58ded
fix(ingest): preserve dbt column name casing (#7063) 2023-01-18 11:57:46 +01:00
Remi
a3f4c40422
feat(ingestion): pull metabase database, schema names from raw query and api (#7039) 2023-01-17 20:06:55 -08:00
Teppo Naakka
87b3a5d0fc
feat(ingest): extract powerbi endorsements to tags (#6638) 2023-01-17 19:47:15 -08:00
Harshal Sheth
cb12910b6b
feat(ingest): add entity registry in codegen (#6984)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-17 19:41:43 -08:00
Mayuri Nehate
7607c04ffa
fix(ingest/kafka): fix ResourceType import error for confluent_kafka<1.9.0 (#7046)
Fixes https://github.com/datahub-project/datahub/issues/7020
2023-01-17 10:33:46 -08:00
Tamas Nemeth
b238272dda
fix(ingest/bigquery): Turning some usage warning message to debug log as it caused confusion (#7024) 2023-01-13 12:30:52 -08:00
John Joyce
b8d8d198c5
feat(ingest): Ingest Previews for Looker Charts, Dashboards, and Explores (#6941) 2023-01-13 10:25:48 -08:00
mohdsiddique
2ae8fe5868
feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion (#7034)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-13 08:31:47 -08:00
Harshal Sheth
9579e69170
fix(ingest/looker): add clarity in chart input parsing logs (#7003) 2023-01-12 20:40:19 -08:00
Teppo Naakka
ad9a5a1832
fix(ingest): powerbi # use display name field as title for powerbi report page (#7017) 2023-01-12 08:12:30 -08:00
mohdsiddique
dcf389d35f
feat(ingestion): Tableau # Embed links (#6994) 2023-01-11 10:57:48 -08:00
Harshal Sheth
93dd87a14b
fix(ingest/snowflake): fix type annotations + refactor get_connect_args (#7004) 2023-01-10 18:47:11 -08:00
Aseem Bansal
2fbdd266f8
fix(ingest): bigquery - views in case more than 1 datasets with views (#6995)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-10 11:22:04 -08:00
Aseem Bansal
c82d4fb2fb
fix(docs): build and broken snowflake docs fix (#6997) 2023-01-10 22:52:36 +05:30
서재권(Data Platform)
9578e418c9
fix(ingest): kafka-connect - support newer version of debezium (#6943)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-09 23:13:00 -08:00
Harshal Sheth
fb758f3867
chore(ingest): finish removing feast-legacy (#6985) 2023-01-09 16:06:14 -08:00
Tamas Nemeth
6fdb19067c
fix(ingest): profiling - Fixing issue with the wrong timestamp stored in check (#6978)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-09 15:50:55 -08:00
dependabot[bot]
78e85398fc
chore(deps): bump certifi from 2020.12.5 to 2022.12.7 in /metadata-ingestion/src/datahub/ingestion/source/feast_image (#6979)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-09 13:16:47 -08:00
Harshal Sheth
432feaa16d
feat(ingest): mark database_alias and env as deprecated (#6901) 2023-01-09 19:58:19 +05:30
danielli-ziprecruiter
0ffb353252
feat(ingest/glue): emit s3 lineage for s3a and s3n schemes (#6788)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-09 07:18:36 -05:00
Aseem Bansal
192aac4271
chore: misc fixes (#6966) 2023-01-09 14:12:36 +05:30
Mayuri Nehate
9f20a23e00
fix(ingest): unfreeze bigquery/snowflake column dataclass (#6921) 2023-01-09 09:07:12 +01:00
VISHAL KUMAR
96ac4c431f
feat(ingest/vertica): support projections and lineage in vertica (#6785)
Co-authored-by: mraman2512 <MY_mramaan2512@gmail.com>
Co-authored-by: Aman.Kumar <64635307+mraman2512@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-06 16:20:19 -05:00
Lucas Roesler
1088182eb5
feat(ingest/postgres): emit lineage for postgres views (#6953) 2023-01-06 15:53:46 -05:00
Harshal Sheth
feaab3b352
fix(ingest/unity): simplify MCP generation and reporting (#6911)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-05 01:47:42 -05:00
Harshal Sheth
f651646d3d
chore(ingest): remove inferred args to MCPW, part 2 (#6905) 2023-01-04 23:29:56 -05:00
Harshal Sheth
8b1dc4bbdf
fix(ingest): use branch info when cloning git repos (#6937) 2023-01-04 16:52:16 -08:00
Fredrik Sannholm
e0aa812621
feat(ingest): allow extracting snowflake tags (#6500) 2023-01-04 16:05:23 -05:00
Harshal Sheth
6bc85502ba
feat(ingest): add include_table_location_lineage flag for SQL common (#6934) 2023-01-04 14:30:33 -05:00
mohdsiddique
54ea8244de
feat(ingestion): PowerBI# Improve PowerBI source ingestion (#6549)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-03 08:08:11 -08:00
cc
4209d6f3dd
fix(ingest/metabase): use card_id in dashboard to chart lineage (#6583)
Co-authored-by: 陈城 <cheng.chen@tenclass.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-30 17:27:09 -05:00
Stijn De Haes
b796db1caf
fix(ingest/airflow): reorder imports to avoid cyclical dependencies (#6719)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-30 13:12:25 -05:00
Harshal Sheth
092d4c808d
fix(cli): fix delete urn cli bug + stricter type annotations (#6903) 2022-12-30 11:36:00 +01:00
Pedro Silva
594fc1bf5a
fix(cli): Make datahub quickstart work with latest docker compose in M1 (#6891)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-30 11:33:18 +01:00
Tamas Nemeth
e81a3ad26d
fix(ingest): profiling (bigquery) - Address biquery profiling query error due to timestamp vs data mismatch (#6874) 2022-12-30 11:32:43 +01:00
Harshal Sheth
dfc5c6bfce
chore(ingest): remove inferred args to MCPW, part 1 (#6819) 2022-12-30 01:26:47 -05:00
Tamas Nemeth
ead0074169
deprecate(ingest): bigquery - Removing bigquery-legacy source (#6851)
Co-authored-by: John Joyce <john@acryl.io>
2022-12-29 13:19:05 -08:00
Marvin Rösch
5167ed40ef
fix(ingest): trino - fall back to default table comment method for all Trino query errors (#6873) 2022-12-29 18:11:21 +01:00
Aseem Bansal
5755d2ca9e
fix(ingest): okta undefined variable error (#6882) 2022-12-29 20:24:22 +05:30
John Joyce
218f3c3414
refactor(docs): Correctly spell elasticsearch in docs (#6880) 2022-12-29 15:21:24 +01:00
Harshal Sheth
667ca8632d
feat(ingest): avoid embedding serialized json in metadata files (#6742) 2022-12-28 19:28:38 -05:00
Harshal Sheth
b474315e07
fix(ingest): conditionally include env in assertion guid (#6811) 2022-12-28 11:35:20 -08:00