Tamas Nemeth
7e5610f358
feat(ingest/dagster): Dagster source ( #10071 )
...
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
2024-03-25 13:28:35 +01:00
david-leifker
f9e64d03cc
feat(auth): view authorization ( #10066 )
2024-03-23 06:15:36 -05:00
RyanHolstien
d55210663f
feeat(openapi): add batch endpoint to v2 using requestbody ( #10100 )
2024-03-21 15:15:29 -05:00
Tamas Nemeth
6eb5f80a9a
Openlineage endpoint and Spark Lineage Beta Plugin ( #9870 )
...
Co-authored-by: David Leifker <david.leifker@acryl.io>
2024-03-01 09:31:21 -06:00
david-leifker
ed10a8d8cc
feat(search): search access controls ( #9892 )
...
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2024-02-28 16:57:26 -06:00
RyanHolstien
131c8f878c
feat(kafka): add health indicator for kafka ( #9662 )
2024-01-19 13:44:48 -06:00
david-leifker
c2bc41d15e
fix(health): fix health check url authentication ( #9117 )
2023-11-03 12:29:31 -05:00
RyanHolstien
1b737243b2
feat(avro): upgrade avro to 1.11 ( #9031 )
2023-10-18 13:45:46 -05:00
david-leifker
56c00f5586
feat(openapi): entity endpoints & analytics raw ( #8537 )
2023-09-07 17:09:52 -05:00
david-leifker
1b79142d9e
feat(EntityService): batched transactions and ebean updates ( #8456 )
2023-09-02 19:25:44 -05:00
RyanHolstien
93fde6bf93
feat(io): refactor metadata-io module ( #8306 )
2023-07-20 06:39:14 +05:30
RyanHolstien
10e5ecf6d2
fix(async): submit additional default aspects only when not in async mode ( #8320 )
2023-06-30 15:56:12 -05:00
david-leifker
cd05f5b174
feat(schema-registry): replace confluent schema registry ( #7930 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-05-01 13:18:41 -05:00
Indy Prentice
3bd637843f
feat(elasticsearch): Add servlet which provides an endpoint for a healthcheck on the ES cl… ( #7799 )
...
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-04-20 11:12:00 -05:00
RyanHolstien
0d5873db2a
feat(patch): patch support for flow info and job info and refactor patchbuilders for java sdk ( #7495 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: David Leifker <david.leifker@acryl.io>
2023-04-13 15:46:35 -05:00
david-leifker
0a9dc73402
build(docker): metadata-ingestion images build and add slim version ( #7412 )
2023-03-20 18:06:35 -05:00
david-leifker
571970cf62
Dockerize updates ( #7387 )
2023-02-22 15:43:11 +05:30
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
david-leifker
be9348b518
fix(build): rename conflicting module auth-api ( #6728 )
...
* fix(build): rename conflicting module `auth-api`
2022-12-09 18:06:51 -06:00
mohdsiddique
f2a83f3074
feat(gms): Pluggable Authentication & Authorization Framework ( #6634 )
2022-12-06 10:22:41 -08:00
mohdsiddique
c31e380489
feat(apache-ranger): Apache Ranger Authorizer support in datahub-gms ( #4999 )
2022-08-01 09:47:08 -07:00
Shirshanka Das
238fca5191
chore: remove unnecessary modules from codebase ( #5420 )
2022-07-17 22:02:45 -07:00
Navin Sharma
ee7c5f75a2
Add Changes to support smoke test on Datahub deployed on kubernetes Cluster ( #5334 )
...
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2022-07-14 22:04:06 +05:30
Tamas Nemeth
c131e13582
feat(airflow): Airflow lineage ingestion plugin ( #4833 )
...
feat(airflow): Airflow lineage ingestion plugin (#4833 )
2022-05-25 09:12:11 +02:00
John Joyce
c69310522b
feat(metadata service): Introducing Platform Events ( #4477 )
2022-03-29 18:32:04 -07:00
leifker
bb8f45bd20
feat(ingestion): add java protobuf schema ingestion ( #4178 )
...
Co-authored-by: leifker <david.leifker@landsend.com>
2022-03-11 10:13:00 -08:00
Ravindra Lanka
1ac20ee764
feat(platform): add timeline api for dataset entity ( #4252 )
2022-02-24 19:40:55 -08:00
John Joyce
271784c9c1
feat(ui): UI-based ingestion (as featured in Dec Townhall) ( #3975 )
2022-01-27 10:33:12 -08:00
Swaroop Jagadish
3e234a992c
feat(java-emitter): improvements to builder API-s, moving spark-lineage under metadata-integration/java ( #3819 )
2022-01-04 01:41:09 -08:00
MugdhaHardikar-GSLab
c65609a84a
feat(datahub-client): add Java REST emitter ( #3781 )
2022-01-02 22:48:38 +05:30
MugdhaHardikar-GSLab
5b68a4dff9
feat(spark-lineage): add ability to push data lineage from spark to d… ( #3664 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-12-13 12:00:51 -08:00
Shirshanka Das
842c05980b
feat(metadata-service): support for custom model extensions without forks ( #3630 )
2021-11-28 21:06:27 -08:00
John Joyce
f49666a230
feat(auth): Metadata Service Authentication! ( #3598 )
2021-11-22 16:33:14 -08:00
Arun Vasudevan
fde42e01e2
Remove GMS GraphQL Service ( #3605 )
2021-11-22 15:53:20 -08:00
John Joyce
a92ab66a3a
refactor(nocode): Final part of No-Code cleanup ( #3477 )
2021-10-31 22:06:36 -07:00
John Joyce
2c5edd88ab
feat(graphql): migrating GraphQL API to metadata-service (nee GMS) ( #3131 )
2021-08-20 10:58:07 -07:00
John Joyce
f3fc0970f3
refactor(build): Remove unnecessary ext modules. ( #3074 )
2021-08-10 22:48:06 -07:00
John Joyce
383bde18f5
chore(frontend): Remove unused files 1/4 ( #3014 )
2021-08-09 09:48:05 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect ( #2983 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Wei Hou
bac1ae42fc
refactor(datahub-web): removing frontend Ember app (i.e. datahub-web folder) ( #2921 )
2021-07-22 15:58:30 -07:00
John Joyce
cc95916201
feat(gms): Merge MAE, MCE consumers into GMS ( #2690 )
2021-06-15 08:44:15 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling ( #2629 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-03 13:24:33 -07:00
Harshal Sheth
d0ca3191c9
build(ingest): add metadata-ingestion to gradle build ( #2510 )
2021-05-06 22:10:49 -07:00
John Joyce
16cada2055
fix(Ember App): Allow ember build (disabled by default) ( #2348 )
...
Co-authored-by: John Joyce <john@acryl.io>
2021-04-06 16:10:50 -07:00
Harshal Sheth
478d232d2f
build: remove deprecated ember app from build ( #2328 )
2021-04-01 12:16:47 -07:00
Harshal Sheth
c015cf7ca9
feat(docs): use gradle for building docs ( #2239 )
2021-03-15 16:13:07 -07:00
Arun Vasudevan
84e952e138
feat (graphql): Datahub GMS Graphql Api Application for Querying Dataset ( #2071 )
2021-02-01 11:51:15 -08:00
Gabe Lyons
e2e3aca478
fix (react): fixing browse routing ( #2069 )
...
* fixing browse routing
* including react app in build
2021-01-27 10:52:46 -08:00
John Joyce
50cec65f57
feat(GQL Queries): Productionalizing GraphQL Part 1: Dataset Query support + adding shared GraphQL module ( #2066 )
...
* Productionalizing GraphQL Part 1: Dataset Query support + introducing common datahub-graphql-core module.
Co-authored-by: John Joyce <jjoyce0510@gmail.com>
2021-01-22 15:44:00 -08:00
John Plaisted
25b663cc18
refactor: move code to linkedin/datahub-gma. ( #1955 )
...
Move code to linkedin/datahub-gma.
"GMA" (Generalized Metadata Architecture) is the backend of DataHub, and has been moved to its own repository.
This deletes the code that was moved and uses jars that GMA publishes to bintray to load it.
Note that not all of GMA was moved, but most of it. We may still move more things to the other repository in the future.
2020-10-23 15:14:57 -07:00
