yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-11-03 20:27:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,944 Commits 408 Branches 129 Tags
Commit Graph

7944 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mayuri Nehate
a0c4e0dd46
feat(ingest): add GCS ingestion source (#7903) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-04-27 19:03:41 +02:00
Harshal Sheth
916cb21454
test(ingest/biz-glossary): add test for enable_auto_id (#7911) 2023-04-26 19:48:52 -07:00
Harshal Sheth
a33153c1f6
feat(sdk): add DataHubGraph.get_entity_semityped method (#7905) 2023-04-26 13:44:13 -07:00
Pedro Silva
967260634c
Revert "feat(cli): Modifies ingest-sample-data command to use DataHub… (#7899) 2023-04-26 16:56:22 +01:00
Harshal Sheth
8f8c11081d
fix(cli): remove duplicate labels from quickstart files (#7886) 2023-04-25 14:35:34 -07:00
Harshal Sheth
29e5cfd643
fix(ingest): fix minor bug + protective dep requirements (#7861) 2023-04-25 14:35:01 -07:00
Mayuri Nehate
031aee4298
fix(ingest/bigquery): fix handling of time decorator offset queries (#7843) 
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
 2023-04-25 13:51:20 -07:00
Mayuri Nehate
ca1f1903ea
fix(ingest/snowflake): fix optimised lineage query, filter temporary tables (#7894) 
With this change, below snowflake query errors for larger lineage time window are fixed:

error 1 - 100099 (22000): Result array of ARRAYAGG is too large.
error 2 - max LOB size (16777216) exceeded, actual size of parsed column is xxxxxxxxxx
 2023-04-25 13:51:04 -07:00
Harshal Sheth
19d7c392d6
feat(sdk): support entity types filter in get_urns_by_filter (#7902) 2023-04-25 13:31:55 -07:00
Harshal Sheth
f314d896b9
docs(airflow): add debug guide on url generation (#7885) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-04-25 12:25:25 -07:00
Harshal Sheth
71ecbd6060
fix(ingest/dbt): ensure dbt shows view properties (#7872) 2023-04-25 12:25:07 -07:00
Mayuri Nehate
28986d8081
fix(ingestion/tableau): backward compatibility with version 2021.1 and above (#7864) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-04-24 11:08:56 -07:00
Mayuri Nehate
3212e74969
feat(ingest/snowflake): optionally emit all upstreams irrespective of recipe pattern (#7842) 2023-04-24 11:01:15 -07:00
Pedro Silva
a5fa933fb0
feat(cli): Modifies ingest-sample-data command to use DataHub url & token based on config (#7896) 2023-04-24 15:52:10 +01:00
Hyejin Yoon
2bc0a781a6
fix: refactor toc (#7862) 2023-04-21 18:36:10 -07:00
Jinlin Yang
a3a8107b92
bug(7882): run kafka-configs.sh on DataHubUpgradeHistory_v1 to make sure the retention.ms is set to infinite (#7883) 2023-04-21 18:31:51 -05:00
Chris Collins
c125384b42
feat(model) Add ContainerPath aspect model (#7774) 2023-04-21 18:37:23 -04:00
david-leifker
2ffa39f009
Logging and test models fixes (#7884) 2023-04-21 15:45:20 -05:00
Andrew Sikowitz
e9c2f9afcc
feat(ingest/unity): Ingest ownership for containers; lookup service principal display names (#7869) 2023-04-21 11:02:39 -07:00
mohdsiddique
f21eeed6e7
feat(ingestion): lookml refinement support (#7781) 
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-04-21 10:55:31 -07:00
Yusuf Mahtab
fa10256c47
feat(glue): allow resource links to be ignored (#7639) 
Co-authored-by: Justas Cernas <justas.cernas@fundingcircle.com>
 2023-04-21 10:42:32 -07:00
Indy Prentice
fa260fc6c8
Remove usages of commons-text library lower than 1.10.0 (#7850) 
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
Co-authored-by: David Leifker <david.leifker@acryl.io>
 2023-04-21 12:30:23 -05:00
Aezo
1a5c716b87
feat(ingest/powerbi): support modified_since, extract_dataset_schema and many more (#7519) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-04-20 22:58:45 -07:00
eeepmb
2616a16ec8
docs(ingest/powerbi): update workspace concept mapping (#7835) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-04-20 22:03:36 -07:00
Harshal Sheth
f37ca4e49c
docs(ingest): fix CorpGroup example (#7816) 2023-04-20 21:09:12 -07:00
Harshal Sheth
66f44945e3
docs(ingest): update dbt and aws docs (#7870) 2023-04-20 21:08:22 -07:00
Pedro Silva
9199c89f13
fix(cache): Fix cache key serialization in search service (#7858) 
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
 2023-04-20 19:07:23 +01:00
Chris Collins
b58894662e
fix(ui) Disable cache on Domain and Glossary Related Entities pages (#7867) 2023-04-20 14:03:25 -04:00
Andrew Sikowitz
1ff6949e36
refactor(ingest): Add helper DataHubGraph methods (#7851) 
Adds:
- get_urns_by_filter(), using scroll by entities
- get_latest_pipeline_checkpoint()
- soft_delete_urn()
 2023-04-20 10:16:33 -07:00
Harry Nguyen
a9a80b8c70
chore(ui): change background color to transparent for avatar with photoUrl (#7527) 
Co-authored-by: Nguyen Tri Hieu <nhieu@fossil.com>
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
 2023-04-20 13:06:39 -04:00
John Joyce
73a12379ae
refactor(platform): Refactoring ES Utils, adding EXISTS condition support to Filter Criterion (#7832) 2023-04-20 09:42:07 -07:00
Chris Collins
90f653b622
fix(ui) Get all entity assertions in chrome extension (#7849) 2023-04-20 12:41:20 -04:00
Chris Collins
34c5847b17
fix(ui) Add UI fixes and design tweaks to AutoComplete (#7845) 2023-04-20 12:28:02 -04:00
Indy Prentice
3bd637843f
feat(elasticsearch): Add servlet which provides an endpoint for a healthcheck on the ES cl… (#7799) 
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
Co-authored-by: Pedro Silva <pedro@acryl.io>
 2023-04-20 11:12:00 -05:00
Indy Prentice
e9364ed1bd
feat(urn) Add validator to reject URNs which contain the character we plan to u… (#7859) 
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
 2023-04-20 11:00:00 -05:00
RyanHolstien
bf86d79d8f
fix(mcl): fix improper pass by reference (#7860) 2023-04-20 10:51:29 -05:00
Aseem Bansal
535e1abe44
chore(ci): fix CI failing due to lint (#7863) 2023-04-20 16:53:36 +05:30
Harshal Sheth
6802142f6e
fix(ingest/salesforce): use report timestamp for operations (#7838) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-04-19 20:39:07 -07:00
Hyejin Yoon
e5d06733f2
feat(docs): consolidate api guides (#7857) 
Co-authored-by: socar-dini <dini@socar.kr>
 2023-04-20 12:17:11 +09:00
Hyejin Yoon
ea4036c1c8
feat: enriching guide on creating dataset (#7777) 
Co-authored-by: Hyejin Yoon <hyejin.yoon@acryl.io>
Co-authored-by: socar-dini <dini@socar.kr>
 2023-04-19 12:58:03 +09:00
Harshal Sheth
f0ea79060b
chore(ingest): bug fix in sqlparse pin (#7848) 2023-04-18 16:05:23 -07:00
Indy Prentice
1b6f6e3a9e
feat(urn): Add a validator when creating an URN that it is no longer than the li… (#7836) 
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
 2023-04-18 22:44:45 +00:00
Harshal Sheth
cf7eb570a0
fix(ingest): pin sqlparse version (#7847) 2023-04-18 14:25:42 -07:00
mohdsiddique
11d6fe2f20
doc(ingestion/tableau): Fix rendering ingestion quickstart guide (#7808) 
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
 2023-04-18 10:39:05 -07:00
John Joyce
b46822399c
feat(timeseries): Support sorting timeseries aspects by non-timestampMillis field + fix operations resolver (#7840) 2023-04-18 09:10:04 -07:00
Aseem Bansal
ee9dfda25e
chore(ci): regular upgrade base requirements.txt (#7821) 2023-04-18 20:36:05 +05:30
Harshal Sheth
399e3333ad
feat(cli): improve quickstart stability (#7839) 2023-04-17 21:19:19 -07:00
Gabe Lyons
80650f05e5
docs(): Update DataJobInputOutput.pdl to express that CLL fields are not shown in the UI right now (#7830) 2023-04-17 19:48:30 -07:00
Jeff Merrick
1c03bef7da
docs(): add markprompt help chat (#7837) 2023-04-17 17:02:56 -07:00
david-leifker
429c54f3ad
fix(ci): fix spark lineage test (#7829) 2023-04-17 15:29:45 -07:00