yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-03 23:28:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,076 Commits 185 Branches 118 Tags
Commit Graph

272 Commits

Author SHA1 Message Date
Mayuri Nehate
5a3f91de53
docs(ingest/bigquery): add permissions to profile google drive backed… (#8490) 2023-07-27 17:01:39 +05:30
Mayuri Nehate
f4fde21168
feat(ingest/nifi): add support for basic auth in nifi (#8457) 2023-07-20 14:12:33 -04:00
Andrew Sikowitz
8a198cd615
fix(ingest/unity): Pin databricks-sdk and update docs (#8293) 2023-06-27 13:38:55 -04:00
Ellie O'Neil
8880b47ca1
docs(business glossary) Update business glossary docs (#8287) 2023-06-26 11:00:09 -07:00
Ellie O'Neil
795b185f59
docs(ingest/lineage): Update fine grained file lineage docs (#8283) 2023-06-23 12:29:54 +05:30
Andrew Sikowitz
9fa8489cb8
feat(ingest/snowflake): Okta OAuth support; update docs (#8157) 2023-06-07 01:09:05 -07:00
Mayuri Nehate
abc2f85331
docs(ingest/nifi): fix broken links (#8143) 2023-05-30 11:04:15 -07:00
Gabe Lyons
ada6ea5a45
docs(csv-enricher): add example csv file & recipe (#8141) 2023-05-29 19:02:26 +05:30
Mayuri Nehate
3e727c5e9c
docs(glue): fix broken link (#8135) 
Co-authored-by: Pedro Silva <pedro@acryl.io>
 2023-05-26 09:25:59 -05:00
Mayuri Nehate
f2c53a3660
feat(ingest/glue): report glue job lineage failures, update doc (#8126) 2023-05-26 10:30:03 +02:00
Andrew Sikowitz
d3cd4dbb0c
feat(ingest/unity): Allow ingestion without metastore admin role (#8091) 
- Adds more detailed docs and connection test
- Fixes empty username queries
 2023-05-24 15:36:22 -07:00
Mayuri Nehate
84270bcac8
feat(ingest/nifi): kerberos authentication (#8097) 
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
Co-authored-by: Indy Prentice <iprentic@users.noreply.github.com>
 2023-05-24 15:09:01 -07:00
matthew-piatkus-cko
bfde4662c7
fix(ingest/salesforce): support JSON web token auth (#7963) 2023-05-05 18:17:43 +00:00
Harshal Sheth
a9e0038199
docs(ingest/postgres): add example with ssl configuration (#7916) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-05-03 15:22:24 -07:00
Andrew Sikowitz
5b290c9bc5
feat(ingest/unity): Add usage extraction; add TableReference (#7910) 
- Adds usage extraction to the unity catalog source and a TableReference object to handle references to tables
Also makes the following refactors:
- Creates UsageAggregator class to usage_common, as I've seen this same logic multiple times.
- Allows customizable user_urn_builder in usage_common as not all unity users are emails. We create emails with a default email_domain config in other connectors like redshift and snowflake, which seems unnecessary now?
- Creates TableReference for unity catalog and adds it to the Table dataclass, for managing string references to tables. Replaces logic, especially in lineage extraction, with these references
- Creates gen_dataset_urn and gen_user_urn on unity source to reduce duplicate code
Breaks up proxy.py into implementation and types
 2023-05-01 11:30:09 -07:00
Mayuri Nehate
a0c4e0dd46
feat(ingest): add GCS ingestion source (#7903) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-04-27 19:03:41 +02:00
Aezo
1a5c716b87
feat(ingest/powerbi): support modified_since, extract_dataset_schema and many more (#7519) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-04-20 22:58:45 -07:00
eeepmb
2616a16ec8
docs(ingest/powerbi): update workspace concept mapping (#7835) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-04-20 22:03:36 -07:00
Harshal Sheth
66f44945e3
docs(ingest): update dbt and aws docs (#7870) 2023-04-20 21:08:22 -07:00
Tamas Nemeth
0cc12bcce7
feat(ingest): redshift - Redshift rework (#6906) 2023-04-12 19:15:43 +02:00
Mayuri Nehate
5fd7981532
fix(ingest/snowflake): fix incorrect tag urn case, improve tag display name (#7758) 2023-04-07 13:07:08 -07:00
Harshal Sheth
82dc2b6393
feat(docs): clear up source configs (#7720) 2023-04-04 18:40:19 +05:30
mohdsiddique
6d6d59141e
feat(ingestion): powerbi # uniquly identify the multiple instance of same platform (#7632) 
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
 2023-03-21 09:27:29 -07:00
mohdsiddique
fc8757d25e
feat(ingestion): powerbi # Amazon Redshift lineage support (#7562) 
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-03-20 10:24:34 -07:00
mohdsiddique
0a36826407
fix(ingest/powerbi): support each expression in m-query function invocation (#7541) 
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-03-15 09:06:29 -07:00
mohdsiddique
de719663ff
feat(ingestion): powerbi # support Google BigQuery table lineage (#7502) 
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-03-07 09:07:00 -08:00
mohdsiddique
29d171106b
feat(ingest/tableau): project path and container support (#7426) 
Co-authored-by: mayurinehate <mayuri.nehate@gslab.com>
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
 2023-03-02 16:53:19 -08:00
Shirshanka Das
07e4d0696f
feat(ingest): json-schema - add json schema support for files and kaf… (#7361) 2023-02-19 08:43:13 -08:00
Tim
ad742f14c8
refactor(ingest,athena): update athena sample recipe (#7368) 2023-02-17 16:24:24 -08:00
mohdsiddique
3a095f960f
feat(ingestion): powerbi # Configurable Admin API (#7055) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
 2023-02-14 09:58:34 -08:00
Mayuri Nehate
e8c141269b
docs(ingest): update kafka connect doc, simplify starter recipe (#7243) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-02-06 16:33:41 +01:00
John Joyce
3d1d5fac7c
refactor(docs): Minor language updates for kafka source doc header (#7237) 2023-02-03 14:42:45 +05:30
Mayuri Nehate
67cb358395
docs(ingest/lineage): clarify name field in entity config for file based lineage (#7225) 2023-02-02 11:33:34 -08:00
mohdsiddique
e34c2ce517
docs(ingestion): powerbi # Add permission for DAX and mashup expressions (#7195) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
 2023-01-31 14:04:36 -08:00
Tamas Nemeth
b84e2ac5b2
fix(doc/biquery): Updating bigquery capability doc (#7136) 2023-01-25 17:19:04 +01:00
Teppo Naakka
2b43bce3d6
feat(ingest): powerbi # add powerbi workspaces to containers (#6532) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-01-24 15:06:50 -08:00
Harshal Sheth
fc41f455a0
feat(ingest): support snapshots in dbt and dbt-cloud (#7062) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-01-18 08:35:03 -08:00
Teppo Naakka
87b3a5d0fc
feat(ingest): extract powerbi endorsements to tags (#6638) 2023-01-17 19:47:15 -08:00
jx2lee
7192f1699d
docs(dbt): fix indentation in dbt meta mapping docs (#7045) 
Co-authored-by: jj.lee <jj.lee@coinone.com>
 2023-01-17 10:35:04 -08:00
Rajasekhar-Vuppala
cd9fc26a25
feat(ingest/vertica): Adding Vertica as source in Datahub UI (#7010) 
Co-authored-by: Vishal <vishal.k@simplify3x.com>
Co-authored-by: VISHAL KUMAR <110387730+vishalkSimplify@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
 2023-01-13 13:23:32 -08:00
Paul Logan
f085ec225b
Docs fixes week of 12 22 (#6963) 
Co-authored-by: John Joyce <john@acryl.io>
 2023-01-06 16:14:49 -08:00
Fredrik Sannholm
e0aa812621
feat(ingest): allow extracting snowflake tags (#6500) 2023-01-04 16:05:23 -05:00
mohdsiddique
54ea8244de
feat(ingestion): PowerBI# Improve PowerBI source ingestion (#6549) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
 2023-01-03 08:08:11 -08:00
Harshal Sheth
e9176d2cd2
docs(ingest/looker): fix typos + update lookml github action example (#6910) 2022-12-30 20:54:43 +01:00
Tamas Nemeth
ead0074169
deprecate(ingest): bigquery - Removing bigquery-legacy source (#6851) 
Co-authored-by: John Joyce <john@acryl.io>
 2022-12-29 13:19:05 -08:00
Mayuri Nehate
2129496c98
feat(ingest/snowflake): handle failures gracefully and raise permission failures (#6748) 2022-12-28 08:20:37 -08:00
mohdsiddique
9daa8ed56f
feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion (#6829) 
* lint fix

* domain in term

* domain in term

* review comments

* add todo

Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2022-12-22 17:47:57 -05:00
Harshal Sheth
e2b4a65a8e
refactor(ingest): clean up exception types (#6818) 2022-12-21 07:28:18 -08:00
orlandine
b219f0848a
docs(ingest/salesforce): list required permissions (#6610) 2022-12-08 14:50:15 -05:00
Mayuri Nehate
eeb7a9dfe5
feat(ingest): snowflake - update snowflake docs, add simple validations (#6636) 2022-12-07 14:56:03 +01:00