yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-08-18 06:06:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,753 Commits 235 Branches 123 Tags
Commit Graph

264 Commits

Author SHA1 Message Date
John Joyce
2c5edd88ab
feat(graphql): migrating GraphQL API to metadata-service (nee GMS) (#3131) 2021-08-20 10:58:07 -07:00
John Joyce
798e57bc24
refactor: remove unused gms code, frontend endpoints part 2/4 (#3015) 
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Dexter Lee <dexter@acryl.io>
 2021-08-17 13:57:01 -07:00
Dexter Lee
872215dc7d
fix(upgrade): Fix MAE consumer and upgrade's dependency issue (#3096) 2021-08-13 10:12:57 -07:00
Dexter Lee
2df9d4fed6
feat: add support for AWS glue schema registry (#3083) 2021-08-11 21:40:37 -07:00
John Joyce
383bde18f5
chore(frontend): Remove unused files 1/4 (#3014) 2021-08-09 09:48:05 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect (#2983) 
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
 2021-07-30 17:41:03 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960) 2021-07-29 20:04:40 -07:00
Dexter Lee
8df8ff96bc
fix(usage-stats): Add usage stats factory to mae processor config (#2803) 2021-06-29 23:03:31 -07:00
John Joyce
09cbc548a4
feat(logs): improve logging in GMS and datahub-frontend (#2761) 2021-06-25 10:56:45 -07:00
Harshal Sheth
937f02c6bc
feat: usage stats (part 1) (#2750) 
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
 2021-06-24 17:11:00 -07:00
Gabe Lyons
3f5e371851
feat(graph): support using elasticsearch as graph backend. (#2726) 2021-06-22 16:08:54 -07:00
Dexter Lee
f3f562fe08
fix(NoCode): Fix product analytics (#2697) 2021-06-15 09:42:16 -07:00
John Joyce
cc95916201
feat(gms): Merge MAE, MCE consumers into GMS (#2690) 2021-06-15 08:44:15 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling (#2629) 
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
 2021-06-03 13:24:33 -07:00
G-nther
634dde5cfb
fix(analytics): use seperate env variable for tracking topic in MAE-Consumer (#2545) 2021-05-12 17:53:40 -07:00
John Joyce
29832e5385
feat(Product Analytics): Introducing In-App Analytics Beta (#2499) 
Co-authored-by: Harshal Sheth <harshal@acryl.io>
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
 2021-05-11 15:41:42 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields (#2228) 
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
 2021-05-10 13:20:23 -07:00
Fredrik Sannholm
e88a671959
Fix(search): fix datajob and dataflow search mappings (#2418) 2021-04-21 12:04:20 -07:00
Dexter Lee
1fc532d831
feat(index): Add index naming convention for elasticsearch (#2386) 2021-04-13 07:56:31 -07:00
shakti-garg
15ed5bb067
feat(kafka-config): add variable KAFKA_CONSUMER_GROUP_ID to override group id value for kafka consumer (#2297) 2021-03-27 20:24:36 -07:00
John Plaisted
5e91014e00
feat(search) BREAKING Support ElasticSearch 7, drop ES5 (#2263) 
Merges in changes from our ES7 branch, and drops support for ES5.

This is a breaking change due to the upgrade, we have a ES5 branch at the commit before this.
 2021-03-18 19:16:44 -07:00
Dexter Lee
8d731b577e
fix(docker): start issue when there are multiple kafka brokers in bootstrap config (#2231) 2021-03-12 11:26:49 -08:00
Gabe Lyons
11e0cd66d4
feat(tag): adding search for tags in gms layer (#2203) 2021-03-10 00:02:58 -08:00
Thomas Larsson
5d6cf95f39
feat(mae-consumer): enable mae-consumer to use ssl when communicating with elasticsearch (#2202) 
Fixes: #2129

Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
 2021-03-09 22:42:04 -08:00
Thomas Larsson
9f4d319596
fix(mae-consumer): mae-consumer needs sslcontext bean (#2199) 
Fixes #2198

Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
 2021-03-09 09:59:54 -08:00
Harshal Sheth
2e8a0cef34
fix(docs): fix broken links (#2186) 2021-03-07 10:46:27 -08:00
Thomas Larsson
8fe9520ddc
feat(datahub-dao): enable services to access gms over https (#2133) 
Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
 2021-02-25 14:26:23 -08:00
RyanHolstien
ea86ade29b
feat: ML Model Backend Implementation (#1896) 
Co-authored-by: RyanHolstien <rholstien@expediagroup.com>
 2021-02-17 13:28:13 -08:00
John Joyce
9e66aa0649
Fixing CorpUsers Search Bug (#2073) 
Co-authored-by: John Joyce <john@acryl.io>
 2021-01-29 12:08:50 -08:00
Nagarjuna Kanamarlapudi
f103998dcb
feat(Search): Inject restli client into index builders (#2024) 
* feat(Search): Inject restli client into index builders
Inject restli client into index builders
 2020-12-03 11:43:48 -08:00
Jyoti Wadhwani
1516a9ad78
Update README.md 2020-11-05 16:12:38 -08:00
Claudio Benfatto
a0bb82673a
fix: add placeholder for logging call parameter (#1968) 
The error was passed as a parameter for the log call but the
lack of the {} placeholder was excluding it from the resulting log line.
 2020-10-28 04:55:40 -07:00
John Plaisted
25b663cc18
refactor: move code to linkedin/datahub-gma. (#1955) 
Move code to linkedin/datahub-gma.

"GMA" (Generalized Metadata Architecture) is the backend of DataHub, and has been moved to its own repository.

This deletes the code that was moved and uses jars that GMA publishes to bintray to load it.

Note that not all of GMA was moved, but most of it. We may still move more things to the other repository in the future.
 2020-10-23 15:14:57 -07:00
Mars Lan
0f68f86114
refactor(gms): reorganize GMS factory namespace (#1916) 
Fixes https://github.com/linkedin/datahub/issues/1658
 2020-10-06 15:21:34 -07:00
Liangjun Jiang
6bb5d24c35
add default KAFKA_BOOTSTRAP_SERVER (#1893) 
Co-authored-by: Liangjun <liajiang@expediagroup.com>
 2020-09-25 19:55:48 -07:00
John Plaisted
821bce7d69
feat: Port mce-cli to Java. (#1871) 
Port mce-cli to Java.

Also moved off the avro format event file to json instead. Much nicer to use :)
 2020-09-25 14:05:29 -07:00
Grant Nicholas
f267c8c8e2
fix(gms): update kafka client libraries to a newer version to support schema registry basic auth + SSL (#1863) 2020-09-09 14:37:47 -07:00
Mars Lan
f4bccaf052
fix(ingestion): set schema registry URL correctly for FMCE producer (#1839) 
Fixes https://github.com/linkedin/datahub/issues/1829
 2020-08-31 04:50:54 -07:00
Javier Sotelo
ad4c80a753
feat(kafka-config): Add ability to configure other Kafka props (#1745) 
* Integarte spring-kafka & spring-boot for security props

- Upgrade spring-kafka to 2.1.14
- Use KafkaListener and KafkaTemplates to enable KafkaAutoConfiguration
- Integrates spring-boot's KafkaProperties into spring-kafka's config

* Cleanup imports

* Add DataHub kafka env vars

* Remove kafka-streams dependency

* Add KafkaProperties to gms; Add docs

* Add to Adoption

* Remove KAFKA_BOOTSTRAP_SERVER default

Co-authored-by: jsotelo <javier.sotelo@viasat.com>
Co-authored-by: Kerem Sahin <ksahin@linkedin.com>
 2020-07-27 05:22:51 -07:00
Jyoti Wadhwani
ad6f1653e1
metadata-models 62.0.3 -> 72.0.8 (#1693) 2020-06-11 10:21:51 -07:00
Kerem Sahin
66271923a7
Update readme for mae-consumer-job 2020-06-09 18:10:29 -07:00
Jyoti Wadhwani
90e449d25e
upgrade the version of neo4j (#1691) 2020-06-04 22:14:57 -07:00
Jyoti Wadhwani
8a8dd3de77
move the gradle plugin version to top level build.gradle (#1685) 2020-06-01 19:00:19 -07:00
Jyoti Wadhwani
2d06e1de5e
Revert "upgrade version of certain dependencies (#1675)" (#1676) 
This reverts commit 81508bcb835df4c46b399fd3029b13259930bcd6.
 2020-05-19 18:31:41 -07:00
Jyoti Wadhwani
81508bcb83
upgrade version of certain dependencies (#1675) 2020-05-19 09:12:21 -07:00
Mars Lan
f932437742
build: start enforcing checkstyle and fix all violations (#1670) 2020-05-11 08:41:02 -07:00
Mars Lan
c43e119b3e
refactor: change the default log location (#1667) 
The original location (/local/log) is not a commonly writable location on Mac & Linux.
The new default location also matches the default data location for docker containers.
 2020-05-08 14:39:20 -07:00
Arun Vasudevan
8c9787c19c
feat: convert MAE application to springboot (#1635) 
* Convert MAE to Spring boot
* Fix after testing
* Changes after testing
* Add file appender for gms and update doc type for ESv5.6
* Review comments
* Fix Review Comments
 2020-04-10 10:23:24 -07:00
Arun Vasudevan
e56e4eafd2
feat: Converting MCE to a Spring boot Application (#1629) 2020-04-06 10:16:41 -07:00
Mars Lan
229272fdc9
Update README.md 2020-02-28 22:08:28 -08:00