* feat(onboarding): adds models and API for onboarding steps feature
* feat(onboarding): adds backend for onboarding steps feature
* feat(onboarding): adds framework and some steps for onboarding steps UI
* feat(ingest): add async option to ingest proposal endpoint
* small tweak to validate before write to K, also keep existing path for timeseries aspects
* avoid double convert
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Move code to linkedin/datahub-gma.
"GMA" (Generalized Metadata Architecture) is the backend of DataHub, and has been moved to its own repository.
This deletes the code that was moved and uses jars that GMA publishes to bintray to load it.
Note that not all of GMA was moved, but most of it. We may still move more things to the other repository in the future.
Requested by a few people in OS. See https://github.com/linkedin/datahub/issues/1840.
Companies need full customization over the topic name. This new class should be easily customizable by using a spring factory.
TODO to finish the implmentation for v5. For right now v5 is LI only and unfinished. Getting this in for v4 so it is useful to other companies now.
TODO AFTER OPEN SOURCE PUSH - make configurable via spring
TODO AFTER SUBMIT - see where else we can use this (jobs, where else?)
Start adding java ETL examples, starting with kafka etl.
We've had a few requests to start providing Java examples rather than Python due to type safety.
I've also started to add these to metadata-ingestion-examples to make it clearer these are *examples*. They can be used directly or as a basis for other things.
As we port to Java we'll move examples to contrib.
Changes include:
Remove all keys that can be moved back to respective GMS
Add support for <, <=, >, >= conditions for the filter API
Update Conditions model for <, <=, >, >= conditions
MP_VERSION=metadata-models:58.0.1
MP_VERSION=wherehows-samza:1.0.56
58.0.1: Remove all keys that can be moved back to respective GMS
58.0.0: Revert "Reverting the commit range: f0c894b490d3df047837cf2fb7b9911c86188cae..4b5f31ed8844f818d7db0880d30c8dc8c7ac0087."
57.0.16: Reverting the commit range: f0c894b490d3df047837cf2fb7b9911c86188cae..4b5f31ed8844f818d7db0880d30c8dc8c7ac0087.
57.0.15: Disable filtering removed entities in browse until META-10900 is solved
57.0.14: (resubmit) add graph index builder for ai-metadata entities and relationships
57.0.13: Reverting the commit range: 830e63b4b40cf701db216952c34d731a7a82ea1d..4255871452062c2fd14651cb4fffb7d337bad300.
57.0.12: add graph index builder for ai-metadata entities and relationships
57.0.11: Fix bug which sets removed field to always true while building DatasetDocument
57.0.10: Change p12 file name to new ina group name
57.0.9: Add removal field in field compliance to flag the proposal as removal or not.
57.0.8: Adding action Builder for DatasetInstance entity
57.0.7: Adding GMA entities and relations for GridWorkflow and GridWorkflowExecution
57.0.6: Adding dataType and dataClassification to the search document
57.0.5: Rename graph entity MlTrainedModel to MlTrainedModelEntity
57.0.4: Code to form the FollowedBy Graph based on the Follow Aspect
57.0.3: add graph entity and relationship models for ai-metadata
57.0.2: Refactor incorrect use of mock in variable names
57.0.1: Add support for <, <=, >, >= conditions for the filter API
57.0.0: Update Conditions model for <, <=, >, >= conditions
56.0.5: update version of pegasus metadata plugin
56.0.4: update container dependency
56.0.3: Move mlFeatures from SnapshotRequestBuilders to ActionRequestbuilder
56.0.2: Adding reserved versions aspect
56.0.1: Create search filter for compliance pending review proposal.
56.0.0: Add Likes aspect resource in metadata restli utils
55.0.6: Fix a bug with getAll API
55.0.5: Move applicable metadata-store SnapshotRequestBuilders to ActionRequestbuilder
55.0.4: EspressoDAO: Updated to expect a separator between entityType and aspectName for config mapping keys
55.0.3: Added EspressoRecordSerializer and EspressoDAOUtils
55.0.2: Rewrote EspressoLocalDAOTest with a mocked EspressoAccessor
55.0.1: Migrate metric-gms SnapshotRequestBuilders to ActionRequestBuilder
55.0.0: [Wormhole] Deprecate Holdem-centric locations in favor of the more general CORP locations, which contain Holdem.
54.0.1: Migrate job-gms SnapshotRequestBuilders to ActionRequestBuilder
wherehows-samza 1.0.56 -> 1.0.56:
1.0.56: Gradle5 migration
MP_VERSION=metadata-models:58.0.1
MP_VERSION=wherehows-samza:1.0.56
This commit is automatically generated by li-opensource tool.
* feat: Enable Failed Metadata Change Event for MCE Processor
No Failed MCE was sent when a MCE was rejected by the system
- Added FailedMetadataChangeEvent kafka topic
- Created FailedMCE event from the MCE record
- Added Kafka Producer to send FailedMCE to the FailedMetadataChangeEvent Topic
* fix: Review Comment
- Added Nonnull Annotations
- Made Local variables as final
* bug: added pegasus2avro test json
- Added pegasus2avro test fmce json
* feat: Add Tests for Event Utils Avro2pegasusfmce
- Added Tests for avro2Pegasusfmce
1.0.34: Downrank inactive users in user search query
1.0.33: Refactor clients to remove snapshot builder
1.0.32: Adding client & integration test for get_all
1.0.30: Implement other clients for corp groups
1.0.28: Add resources for search and autocomplete for corp groups
1.0.27: Start using BaseClient from metadata-models
1.0.26: Add get_all resource for CorpUsers
metadata-models 38.1.12 -> 50.0.6:
50.0.2: Fix removed field update logic for all entities
49.0.1: Add dataset graph builder with DownstreamOf relationship
48.0.3: support query dao with traverse paths
47.0.2: refactor the query dao with relationship filter model
47.0.1: Fix for creating duplicate nodes when label for the node is missing
46.0.21: extend filter model with relationship direction
46.0.19: add unit test for entities partial update
46.0.16: Allow relationship filter in the model and query dao
46.0.15: support relationship directions for multi hop query
46.0.14: Implementing reportsto relationship builder and corpuser graph builder
46.0.10: refactor query dao interface using nullable to replace optional
46.0.9: Rename Mock Utils to Test Utils in Metadata-models mp
46.0.6: Remove search index config from metadata models
46.0.2: neo4j query DAO with relationships directions support
45.1.7: refactoring the graph relationship builders
45.1.5: Use correct total count in search response
45.1.3: Fix issue with empty search query filter
45.1.2: Fix a bug with autocomplete limit param
45.0.3: Change platform field type in the dataset search document
45.0.2: implement multi hops query DAO with interface 5
45.0.1: Moving dataset browsePaths build logic from wherehows-samza
44.0.2: implement interface 2 in query DAO
40.0.2: Only return records which exist in the DB after getting search hits
39.0.0: Add a getAuditor method to BaseSnapshotResource rather than taking it in as a constructor argument
38.1.13: Move BaseClient to metadata-models out of GMS template
38.1.12: Remove default filtering on removed field for get_all
MP_VERSION=corp-identity-gms:1.0.40
MP_VERSION=metadata-models:50.0.6
MP_VERSION=wherehows-samza:1.0.56
This commit is automatically generated by li-opensource tool.
