3904 Commits

Author SHA1 Message Date
mohdsiddique
2ae8fe5868
feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion (#7034)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-13 08:31:47 -08:00
Harshal Sheth
f0432ee101
chore(ingest): remove duplicate data_platform.json file (#7026) 2023-01-12 20:47:08 -08:00
Harshal Sheth
9579e69170
fix(ingest/looker): add clarity in chart input parsing logs (#7003) 2023-01-12 20:40:19 -08:00
Teppo Naakka
ad9a5a1832
fix(ingest): powerbi # use display name field as title for powerbi report page (#7017) 2023-01-12 08:12:30 -08:00
mohdsiddique
dcf389d35f
feat(ingestion): Tableau # Embed links (#6994) 2023-01-11 10:57:48 -08:00
Harshal Sheth
ff49d943bc
fix(ingest): remove dead code from tests (#7005)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-11 10:53:05 -08:00
Harshal Sheth
93dd87a14b
fix(ingest/snowflake): fix type annotations + refactor get_connect_args (#7004) 2023-01-10 18:47:11 -08:00
Aseem Bansal
2fbdd266f8
fix(ingest): bigquery - views in case more than 1 datasets with views (#6995)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-01-10 11:22:04 -08:00
Aseem Bansal
c82d4fb2fb
fix(docs): build and broken snowflake docs fix (#6997) 2023-01-10 22:52:36 +05:30
서재권(Data Platform)
9578e418c9
fix(ingest): kafka-connect - support newer version of debezium (#6943)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-09 23:13:00 -08:00
Harshal Sheth
fb758f3867
chore(ingest): finish removing feast-legacy (#6985) 2023-01-09 16:06:14 -08:00
Tamas Nemeth
6fdb19067c
fix(ingest): profiling - Fixing issue with the wrong timestamp stored in check (#6978)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-09 15:50:55 -08:00
dependabot[bot]
78e85398fc
chore(deps): bump certifi from 2020.12.5 to 2022.12.7 in /metadata-ingestion/src/datahub/ingestion/source/feast_image (#6979)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-01-09 13:16:47 -08:00
Harshal Sheth
432feaa16d
feat(ingest): mark database_alias and env as deprecated (#6901) 2023-01-09 19:58:19 +05:30
danielli-ziprecruiter
0ffb353252
feat(ingest/glue): emit s3 lineage for s3a and s3n schemes (#6788)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-09 07:18:36 -05:00
Aseem Bansal
192aac4271
chore: misc fixes (#6966) 2023-01-09 14:12:36 +05:30
Mayuri Nehate
9f20a23e00
fix(ingest): unfreeze bigquery/snowflake column dataclass (#6921) 2023-01-09 09:07:12 +01:00
Paul Logan
f085ec225b
Docs fixes week of 12 22 (#6963)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-06 16:14:49 -08:00
Harshal Sheth
211c30fe30
fix(ingest): add missing dep for powerbi (#6969) 2023-01-06 18:16:32 -05:00
VISHAL KUMAR
96ac4c431f
feat(ingest/vertica): support projections and lineage in vertica (#6785)
Co-authored-by: mraman2512 <MY_mramaan2512@gmail.com>
Co-authored-by: Aman.Kumar <64635307+mraman2512@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-01-06 16:20:19 -05:00
Lucas Roesler
1088182eb5
feat(ingest/postgres): emit lineage for postgres views (#6953) 2023-01-06 15:53:46 -05:00
Aseem Bansal
d55ad6ca14
fix(ci): restrict GE to fix build issues (#6967) 2023-01-06 18:25:36 +05:30
Harshal Sheth
feaab3b352
fix(ingest/unity): simplify MCP generation and reporting (#6911)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-05 01:47:42 -05:00
Harshal Sheth
f651646d3d
chore(ingest): remove inferred args to MCPW, part 2 (#6905) 2023-01-04 23:29:56 -05:00
Harshal Sheth
8b1dc4bbdf
fix(ingest): use branch info when cloning git repos (#6937) 2023-01-04 16:52:16 -08:00
Harshal Sheth
9bb1c155bd
chore(ingest): partially revert pyspark dep from #6908 (#6954) 2023-01-04 16:51:44 -08:00
Harshal Sheth
e97903f7f6
chore(ingest): unpin pydantic dep (#6909) 2023-01-04 16:31:04 -08:00
Fredrik Sannholm
e0aa812621
feat(ingest): allow extracting snowflake tags (#6500) 2023-01-04 16:05:23 -05:00
Harshal Sheth
6bc85502ba
feat(ingest): add include_table_location_lineage flag for SQL common (#6934) 2023-01-04 14:30:33 -05:00
mohdsiddique
54ea8244de
feat(ingestion): PowerBI# Improve PowerBI source ingestion (#6549)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2023-01-03 08:08:11 -08:00
cc
4209d6f3dd
fix(ingest/metabase): use card_id in dashboard to chart lineage (#6583)
Co-authored-by: 陈城 <cheng.chen@tenclass.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-30 17:27:09 -05:00
Harshal Sheth
e9176d2cd2
docs(ingest/looker): fix typos + update lookml github action example (#6910) 2022-12-30 20:54:43 +01:00
Harshal Sheth
b9677229a1
chore(ingest): loosen pyspark and pydeequ deps (#6908) 2022-12-30 20:53:38 +01:00
Harshal Sheth
62a2aa94f6
feat: remove jq requirement + tweak modeldocgen args (#6904)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-30 14:02:57 -05:00
Stijn De Haes
b796db1caf
fix(ingest/airflow): reorder imports to avoid cyclical dependencies (#6719)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-30 13:12:25 -05:00
Harshal Sheth
092d4c808d
fix(cli): fix delete urn cli bug + stricter type annotations (#6903) 2022-12-30 11:36:00 +01:00
Pedro Silva
594fc1bf5a
fix(cli): Make datahub quickstart work with latest docker compose in M1 (#6891)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-30 11:33:18 +01:00
Tamas Nemeth
e81a3ad26d
fix(ingest): profiling (bigquery) - Address biquery profiling query error due to timestamp vs data mismatch (#6874) 2022-12-30 11:32:43 +01:00
Harshal Sheth
1b889022f0
test(ingest/kafka-connect): make docker setup more reliable (#6902) 2022-12-30 11:31:33 +01:00
Harshal Sheth
dfc5c6bfce
chore(ingest): remove inferred args to MCPW, part 1 (#6819) 2022-12-30 01:26:47 -05:00
Tamas Nemeth
ead0074169
deprecate(ingest): bigquery - Removing bigquery-legacy source (#6851)
Co-authored-by: John Joyce <john@acryl.io>
2022-12-29 13:19:05 -08:00
Marvin Rösch
5167ed40ef
fix(ingest): trino - fall back to default table comment method for all Trino query errors (#6873) 2022-12-29 18:11:21 +01:00
Aseem Bansal
5755d2ca9e
fix(ingest): okta undefined variable error (#6882) 2022-12-29 20:24:22 +05:30
John Joyce
218f3c3414
refactor(docs): Correctly spell elasticsearch in docs (#6880) 2022-12-29 15:21:24 +01:00
Aseem Bansal
b8664d6630
fix(lint): pin pydantic version (#6886) 2022-12-29 19:36:14 +05:30
Harshal Sheth
667ca8632d
feat(ingest): avoid embedding serialized json in metadata files (#6742) 2022-12-28 19:28:38 -05:00
Harshal Sheth
b474315e07
fix(ingest): conditionally include env in assertion guid (#6811) 2022-12-28 11:35:20 -08:00
Mayuri Nehate
2129496c98
feat(ingest/snowflake): handle failures gracefully and raise permission failures (#6748) 2022-12-28 08:20:37 -08:00
Tamas Nemeth
25b5a12b9d
feat(ingest): bigquery/snowflake - Store last profile date in state (#6832) 2022-12-28 12:09:18 +01:00
cccs-eric
ec8a4e0eab
feat(ingest): upgrade pydantic version (#6858)
This PR also removes the requirement on docker-compose v1 and makes our tests use v2 instead.

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-27 17:06:16 -05:00