5186 Commits

Author SHA1 Message Date
Mars Lan
468b66fde3
docs: update quickstart debug guide 2020-04-06 07:19:57 -07:00
Mars Lan
d0487239b2
docs: add instructions on how to nuke docker environment 2020-04-05 16:03:29 -07:00
Mars Lan
5d180b43f1
doc: update next town hall date 2020-04-05 11:56:07 -07:00
Bharat
e7feaf1fc0
fix: Reduce mae-consumer & mce-consumer docker image sizes (#1632) 2020-04-05 09:30:05 -07:00
Kerem Sahin
a28f4db5e0
fix(Docker): Fixing 'dockerize not found' issue while starting datahub-gms container using quickstart (#1631) 2020-04-04 13:34:28 -07:00
Bharat
51c0a5c6fe
fix: Reduce gms & frontend docker image sizes (#1630)
* fix: Reduce gms docker image size to 254MB from 1.15GB
* fix: Reduce frontend docker image size to 294MB from 2.68GB
2020-04-04 07:51:29 -07:00
Jyoti Wadhwani
5895ebb3ca
Update townhalls.md 2020-04-03 15:39:27 -07:00
Jyoti Wadhwani
1caad3b053
Update townhalls.md
Add link to BJ
2020-04-03 15:37:36 -07:00
Jyoti Wadhwani
7de7bf89b0
Update search-onboarding.md
Replace with the new path for CorpUsers resource
2020-04-03 08:26:30 -07:00
Jyoti Wadhwani
db8a939910
Update graph-onboarding.md
Replace with the new path for DownstreamLineageResource subresource
2020-04-03 08:24:57 -07:00
Mars Lan
980f39ae88
refactor(frontend): fails loudly to help debug gms issue (#1626)
This will print the full stacktrace in log to help identify various gms issues
2020-04-03 07:40:07 -07:00
Mars Lan
c3c6212d4e
Update debugging.md 2020-04-03 07:36:12 -07:00
Mars Lan
6370251bf0
fix(gms): unable to find registered resources (#1625)
Fix the unable to find registered resource error introduced by 6a309332f321fc6847a721629c276e62867bf52c
Also simplify the build script
2020-04-02 20:53:22 -07:00
Mars Lan
a063ec7297
build(gms): rename JettyRunWar task to run (#1624)
Also make sure the task depends on build so it'll rebuild if needed.
This make it more consistent with other modules such as mce-consumer-job.
2020-04-02 20:16:00 -07:00
Mars Lan
aa81e774fd
doc: fix example MCEs 2020-04-02 19:39:12 -07:00
Mars Lan
4a77576d31
build: pegasus plugin doesn't work well with gradle caching (#1622)
Snapshot & IDL will not be regenerated consistently when gradle caching is enabled.
2020-04-02 19:27:24 -07:00
Mars Lan
bd00b2b120
Update add-new-aspect.md 2020-04-02 19:09:24 -07:00
Jyoti Wadhwani
2555d0a152
Update debugging.md 2020-04-02 16:52:25 -07:00
Mars Lan
d150157d6f
Update developers.md 2020-04-02 07:48:37 -07:00
Mars Lan
6cb3f94b82
fix: elasticsearch not starting on Mac (#1621)
The `chown 1000:1000` trick mentioned in https://discuss.elastic.co/t/elastic-elasticsearch-docker-not-assigning-permissions-to-data-directory-on-run/65812/4 only works on Linux. Make the directory world writeable instead so that it works on both Mac & Linux.
2020-04-02 07:19:52 -07:00
Mars Lan
eb124f839d
Update debugging.md 2020-04-02 07:00:46 -07:00
Mars Lan
9773077fe2
Update README.md 2020-04-02 06:25:39 -07:00
Jyoti Wadhwani
6a309332f3
refactor(gms): move gms restli resources (#1619)
Move dataset and identity resources and subresources to gms/impl/src/main/java/com/linkedin/metadata/resources
Update docs to reflect the change
2020-04-01 16:30:59 -07:00
Mars Lan
7dba331f36
doc: update new entity onboarding instructions 2020-04-01 11:25:03 -07:00
Mars Lan
36beab7da6
refactor: move rest.li keys from metadata-models to gms where they belong (#1617) 2020-04-01 11:02:39 -07:00
Mars Lan
df325ec37e
refactor: remove unused legacy model & move DatasetSearchConfig (#1618) 2020-04-01 10:41:03 -07:00
Kerem Sahin
0ff6e14ff3
Update add-new-aspect.md 2020-04-01 08:42:27 -07:00
Mars Lan
0253f1f684
doc: add rest.li endpoint example 2020-04-01 07:16:20 -07:00
Mars Lan
a76a8f834c
Update add-new-aspect.md 2020-04-01 07:00:44 -07:00
Mars Lan
e69b83442d
doc: additional instruction on gms endpoitns 2020-04-01 07:00:05 -07:00
Mars Lan
72ef018070
doc: add how-to guide on adding a new aspect 2020-03-31 21:48:56 -07:00
Jyoti Wadhwani
2fed2e8147
metadata-models 58.0.1 -> 62.0.3 (#1616)
Expose EbeanServer from EbeanLocalDAO
Create test URNs and use those from entity models
Implement interface for browse

MP_VERSION=metadata-models:62.0.3
MP_VERSION=wherehows-samza:1.0.56
2020-03-31 16:24:52 -07:00
Jyoti Wadhwani
bca0c35030
Update the agenda for April 3rd 2020-03-31 09:20:20 -07:00
Jyoti Wadhwani
47d8d83af4
Update faq.md 2020-03-30 22:24:45 -07:00
Mars Lan
e53975b5d5
doc: add Q&A link 2020-03-30 15:05:54 -07:00
Mars Lan
10f7c1205b
doc: styling tweak 2020-03-29 08:04:17 -07:00
Mars Lan
dd9895e437
doc: minor styling tweak 2020-03-29 08:02:53 -07:00
Mars Lan
0601ad889a
doc: add quick links to root README 2020-03-29 07:59:59 -07:00
larluo
ad91a528ce
add oracle driver (#1613) 2020-03-27 18:28:55 -07:00
Chris Lee
ba33c7a5cd Specify python version in mce-cli requirement.txt 2020-03-27 13:33:22 -07:00
Chris Lee
d1cf62854d
Fix: Docker Quickstart - Sample Data Loading Error
Specify the python version for the required confluent-Kafka library.
2020-03-27 13:14:23 -07:00
Jyoti Wadhwani
e48c1d89d5
Update faq.md with another question
How is metadata ingested in Datahub? Is it real-time?
2020-03-27 10:21:03 -07:00
Mars Lan
e2eac45b85
refactor: remove unused legacy models (#1614) 2020-03-27 10:17:06 -07:00
Jyoti Wadhwani
5fb3c97315
metadata-models 54.0.1 -> 58.0.1 (#1610)
Changes include:
Remove all keys that can be moved back to respective GMS
Add support for <, <=, >, >= conditions for the filter API
Update Conditions model for <, <=, >, >= conditions

MP_VERSION=metadata-models:58.0.1
MP_VERSION=wherehows-samza:1.0.56
2020-03-26 21:53:29 -07:00
larluo
c2e4761753
init jdbc generator (#1611) 2020-03-26 21:32:26 -07:00
Bharat
502db03b0d
add support for kubernetes helm packaging (#1609) 2020-03-26 19:28:30 -07:00
larluo
3fae73691e
add lineage hive (#1608) 2020-03-25 21:54:22 -07:00
Mars Lan
38fc7249d2 Revert "metadata-models 54.0.1 -> 58.0.1:"
This reverts commit bab5daa56d77f067de949f6f0eb5bc7c537641f7.
2020-03-25 21:43:28 -07:00
Jyoti Wadhwani
bab5daa56d metadata-models 54.0.1 -> 58.0.1:
58.0.1: Remove all keys that can be moved back to respective GMS
    58.0.0: Revert "Reverting the commit range: f0c894b490d3df047837cf2fb7b9911c86188cae..4b5f31ed8844f818d7db0880d30c8dc8c7ac0087."
   57.0.16: Reverting the commit range: f0c894b490d3df047837cf2fb7b9911c86188cae..4b5f31ed8844f818d7db0880d30c8dc8c7ac0087.
   57.0.15: Disable filtering removed entities in browse until META-10900 is solved
   57.0.14: (resubmit) add graph index builder for ai-metadata entities and relationships
   57.0.13: Reverting the commit range: 830e63b4b40cf701db216952c34d731a7a82ea1d..4255871452062c2fd14651cb4fffb7d337bad300.
   57.0.12: add graph index builder for ai-metadata entities and relationships
   57.0.11: Fix bug which sets removed field to always true while building DatasetDocument
   57.0.10: Change p12 file name to new ina group name
    57.0.9: Add removal field in field compliance to flag the proposal as removal or not.
    57.0.8: Adding action Builder for DatasetInstance entity
    57.0.7: Adding GMA entities and relations for GridWorkflow and GridWorkflowExecution
    57.0.6: Adding dataType and dataClassification to the search document
    57.0.5: Rename graph entity MlTrainedModel to MlTrainedModelEntity
    57.0.4: Code to form the FollowedBy Graph based on the Follow Aspect
    57.0.3: add graph entity and relationship models for ai-metadata
    57.0.2: Refactor incorrect use of mock in variable names
    57.0.1: Add support for <, <=, >, >= conditions for the filter API
    57.0.0: Update Conditions model for <, <=, >, >= conditions
    56.0.5: update version of pegasus metadata plugin
    56.0.4: update container dependency
    56.0.3: Move mlFeatures from SnapshotRequestBuilders to ActionRequestbuilder
    56.0.2: Adding reserved versions aspect
    56.0.1: Create search filter for compliance pending review proposal.
    56.0.0: Add Likes aspect resource in metadata restli utils
    55.0.6: Fix a bug with getAll API
    55.0.5: Move applicable metadata-store SnapshotRequestBuilders to ActionRequestbuilder
    55.0.4: EspressoDAO: Updated to expect a separator between entityType and aspectName for config mapping keys
    55.0.3: Added EspressoRecordSerializer and EspressoDAOUtils
    55.0.2: Rewrote EspressoLocalDAOTest with a mocked EspressoAccessor
    55.0.1: Migrate metric-gms SnapshotRequestBuilders to ActionRequestBuilder
    55.0.0: [Wormhole] Deprecate Holdem-centric locations in favor of the more general CORP locations, which contain Holdem.
    54.0.1: Migrate job-gms SnapshotRequestBuilders to ActionRequestBuilder
wherehows-samza 1.0.56 -> 1.0.56:

    1.0.56: Gradle5 migration
MP_VERSION=metadata-models:58.0.1
MP_VERSION=wherehows-samza:1.0.56

This commit is automatically generated by li-opensource tool.
2020-03-25 21:13:14 -07:00
Bharat
bfc6c6fd0b
feat: support for Kubernetes-based deployment (#1602)
* Support for Kubernetes based deployments

* Update README.md

* Updated the README to change the configmap with relevant docker host details

* Fixed README based on the comments in PR

* Fixed README to point to Confluent Kafka Helm charts

Co-authored-by: Mars Lan <mars.th.lan@gmail.com>
2020-03-24 06:01:51 -07:00