yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-13 12:10:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,076 Commits 194 Branches 118 Tags
Commit Graph

1661 Commits

Author SHA1 Message Date
Mayuri Nehate
2cffec9452
fix(check upgrade): update logic to compare server and client version (#7238) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-02-13 13:09:38 -08:00
Andrew Sikowitz
8901498582
fix(transformers): pattern add domain transformer - enable replace_existing (#7317) 2023-02-13 12:52:44 -08:00
Tamas Nemeth
f10d622e47
fix(ingest/bigquery): Improve memory usage of lineage extraction (#7326) 2023-02-13 19:59:11 +01:00
Tamas Nemeth
b34e4fe1f1
fix(ingest/bigquery): Fix for table cache was not cleared (#7323) 2023-02-13 19:04:19 +01:00
Harshal Sheth
55442042ff
feat(cli): improve startup time (#7292) 2023-02-10 21:36:01 +05:30
Aseem Bansal
ca65f9db18
refactor(github): change github reference to git references (#7308) 2023-02-10 21:07:44 +05:30
Tamas Nemeth
1402071e48
fix(ingest/bigquery) - Fix for Bigquery parser quoted semicolon in the FROM table name as well (#7277) 2023-02-08 10:18:55 +01:00
Aditya Radhakrishnan
bd17dde7b0
fix(ingest-idp): emit empty GroupMembership when there are no groups (#7196) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-02-02 11:56:40 -08:00
Daniel Messias
0d67e188ef
feat(glue): Use table name as human-readable name for Glue ingestion (#7213) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-02-02 18:04:35 +01:00
Dmitry Bryazgin
1a1f2906db
feat(ingest): Add custom properties to the ldap ingestion (#7125) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-02-01 16:37:14 -08:00
Harshal Sheth
db1a0f13f3
fix(ingest): fix issue in glue tests (#7185) 2023-01-30 21:51:21 -08:00
Harshal Sheth
670342ab64
feat(ingest): switch dbt to use auto_stale_entity_removal (#7160) 2023-01-30 19:59:20 -08:00
mohdsiddique
54562cd2d5
fix(ingestion/powerbi): Not all arguments converted to string (#7157) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
 2023-01-27 18:49:27 +01:00
Harshal Sheth
927d45dda9
feat(ingest): add --log-file option and show CLI logs in UI report (#7118) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-01-26 09:25:02 -08:00
Teppo Naakka
2b43bce3d6
feat(ingest): powerbi # add powerbi workspaces to containers (#6532) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-01-24 15:06:50 -08:00
Harshal Sheth
45f50d2614
test(ingest): fix kafka admin client mocking (#7098) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-01-23 16:22:20 +01:00
Tamas Nemeth
0cdb5e4b4b
refactor(ingest/containers): Refactoring container creation to common place (#6877) 2023-01-21 00:14:31 +01:00
Harshal Sheth
e23eb7108f
feat(ingest): reporting revamp, part 1 (#7031) 2023-01-18 13:34:32 -08:00
Harshal Sheth
d7aa61285b
fix(ingest): support git clone of non-github repos (#7065) 2023-01-18 13:30:24 -08:00
Harshal Sheth
35bd73a28b
feat(ingest): fix handling of unions with aliases in post restli conversion (#7058) 2023-01-18 09:29:46 -08:00
Tim
e2ad881d79
refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location (#7044) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-01-18 09:25:37 -08:00
Harshal Sheth
fc41f455a0
feat(ingest): support snapshots in dbt and dbt-cloud (#7062) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-01-18 08:35:03 -08:00
Mayuri Nehate
4e7faa5503
fix(ingest/tableau): fix node limit exceeded error for workbooks query (#7068) 2023-01-18 14:34:40 +01:00
Teppo Naakka
87b3a5d0fc
feat(ingest): extract powerbi endorsements to tags (#6638) 2023-01-17 19:47:15 -08:00
Harshal Sheth
cb12910b6b
feat(ingest): add entity registry in codegen (#6984) 
Co-authored-by: Pedro Silva <pedro@acryl.io>
 2023-01-17 19:41:43 -08:00
Harshal Sheth
890dae0199
fix(ingest): temporarily disable vertica tests (#7059) 2023-01-17 12:37:16 -08:00
John Joyce
b8d8d198c5
feat(ingest): Ingest Previews for Looker Charts, Dashboards, and Explores (#6941) 2023-01-13 10:25:48 -08:00
mohdsiddique
2ae8fe5868
feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion (#7034) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
 2023-01-13 08:31:47 -08:00
Teppo Naakka
ad9a5a1832
fix(ingest): powerbi # use display name field as title for powerbi report page (#7017) 2023-01-12 08:12:30 -08:00
mohdsiddique
dcf389d35f
feat(ingestion): Tableau # Embed links (#6994) 2023-01-11 10:57:48 -08:00
Harshal Sheth
ff49d943bc
fix(ingest): remove dead code from tests (#7005) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-01-11 10:53:05 -08:00
Harshal Sheth
432feaa16d
feat(ingest): mark database_alias and env as deprecated (#6901) 2023-01-09 19:58:19 +05:30
VISHAL KUMAR
96ac4c431f
feat(ingest/vertica): support projections and lineage in vertica (#6785) 
Co-authored-by: mraman2512 <MY_mramaan2512@gmail.com>
Co-authored-by: Aman.Kumar <64635307+mraman2512@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-01-06 16:20:19 -05:00
Harshal Sheth
f651646d3d
chore(ingest): remove inferred args to MCPW, part 2 (#6905) 2023-01-04 23:29:56 -05:00
Harshal Sheth
8b1dc4bbdf
fix(ingest): use branch info when cloning git repos (#6937) 2023-01-04 16:52:16 -08:00
Fredrik Sannholm
e0aa812621
feat(ingest): allow extracting snowflake tags (#6500) 2023-01-04 16:05:23 -05:00
mohdsiddique
54ea8244de
feat(ingestion): PowerBI# Improve PowerBI source ingestion (#6549) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
 2023-01-03 08:08:11 -08:00
Harshal Sheth
1b889022f0
test(ingest/kafka-connect): make docker setup more reliable (#6902) 2022-12-30 11:31:33 +01:00
Harshal Sheth
dfc5c6bfce
chore(ingest): remove inferred args to MCPW, part 1 (#6819) 2022-12-30 01:26:47 -05:00
Tamas Nemeth
ead0074169
deprecate(ingest): bigquery - Removing bigquery-legacy source (#6851) 
Co-authored-by: John Joyce <john@acryl.io>
 2022-12-29 13:19:05 -08:00
Harshal Sheth
667ca8632d
feat(ingest): avoid embedding serialized json in metadata files (#6742) 2022-12-28 19:28:38 -05:00
Mayuri Nehate
2129496c98
feat(ingest/snowflake): handle failures gracefully and raise permission failures (#6748) 2022-12-28 08:20:37 -08:00
cccs-eric
ec8a4e0eab
feat(ingest): upgrade pydantic version (#6858) 
This PR also removes the requirement on docker-compose v1 and makes our tests use v2 instead.

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2022-12-27 17:06:16 -05:00
Harshal Sheth
31260888fc
feat(ingest/airflow): support raw dataset urns in airflow lineage (#6854) 
* feat(ingest/airflow): support dataset Urns in airflow lineage

This PR also
- resolves a reported circular import issue
- refactors the Airflow tests to reduce duplication

* fix test
 2022-12-27 08:59:26 +01:00
Mayuri Nehate
69a2347db1
feat(ingest): update profiling to fetch configurable number of sample values (#6859) 2022-12-27 08:57:26 +01:00
mohdsiddique
9daa8ed56f
feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion (#6829) 
* lint fix

* domain in term

* domain in term

* review comments

* add todo

Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2022-12-22 17:47:57 -05:00
Mayuri Nehate
a05c5c4069
feat(ingest): extract kafka topic config properties as customProperties (#6783) 2022-12-22 09:34:55 +01:00
John Joyce
2e3a25123d
refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker (#6002) 2022-12-21 08:02:59 -08:00
Harshal Sheth
e2b4a65a8e
refactor(ingest): clean up exception types (#6818) 2022-12-21 07:28:18 -08:00
Harshal Sheth
137f4500b6
feat(ingest/stateful): remove platform_instance_id from state urn (#6795) 2022-12-20 12:12:19 -05:00