397 Commits

Author SHA1 Message Date
Fredrik Sannholm
da6b3d111d
feat(datajob): Backend implementation (#2197) 2021-03-13 08:00:44 -08:00
Gabe Lyons
11e0cd66d4
feat(tag): adding search for tags in gms layer (#2203) 2021-03-10 00:02:58 -08:00
Gabe Lyons
92387019ff
feat(react): adding ability to support theming of datahub, with two themes included (#2193) 2021-03-09 23:14:52 -08:00
Harshal Sheth
d220647094
feat: add date and time types to SQL model (#2201) 2021-03-09 23:07:20 -08:00
Gabe Lyons
adfe60e97a
feat(tags): adding support for read/write of tags in gms & read-only in react datahub-frontend. (#2164) 2021-03-07 11:26:47 -08:00
John Joyce
4dcea8c1d3
feat(gms): Add optional data platform display name (#2148)
Co-authored-by: John Joyce <john@acryl.io>
2021-02-26 21:22:18 -08:00
Gabe Lyons
e6f7f39d15
docs(Ownership): making lack of support for ownergroups in frontend explicit in pdl (#2139) 2021-02-24 22:32:55 -08:00
John Joyce
4f8d8b31ac
feat: Introducing optional DataPlatform logo url (#2127)
Co-authored-by: John Joyce <john@acryl.io>
2021-02-19 19:51:49 -08:00
RyanHolstien
ea86ade29b
feat: ML Model Backend Implementation (#1896)
Co-authored-by: RyanHolstien <rholstien@expediagroup.com>
2021-02-17 13:28:13 -08:00
Nagarjuna Kanamarlapudi
f9d33f5519
(refactor): Convert dataPlatforms to GMA aspect models and associated resource to GMA resource. (#2057)
* (refactor): Convert dataPlatforms to GMA aspect and associated resource to GMA resource.

BREAKING CHANGE: /datasets/dataPlatforms API is now changed to become GMA resource.

* Change documentation style
2021-01-20 15:50:48 -08:00
Harsh Shah
06cca77a6b
feat(models): Add DataFlow and DataJob models (#2008)
Add DataFlow and DataJob urn and aspect models.
The aspects include metadata associated with Data processing flow and job entities,
associated inputs and outputs of a job.
2020-11-30 17:48:29 -08:00
Kerem Sahin
4d8320e4a0
feat(dashboard): Dashboards backend implementation (#1884) 2020-11-23 09:25:58 -08:00
Nagarjuna Kanamarlapudi
5d083143db
feat(dataset): Enable search of datasets by field names (#2001)
* feat(dataset): Enable search of datasets by field names
2020-11-20 12:01:07 -08:00
John Plaisted
6f59a91865
Bump GMA to latest (#2007)
Involves fixing some types. Bright side is that ModelValidation actually checks aspects now (it did not before because aspects are not records but unions!).
2020-11-18 14:07:48 -08:00
Kerem Sahin
733893f5f9
feat(dashboard): Dashboard models update (#1932)
* feat(dashboard): Dashboard models update

* Keep chartId/dashboardId fields in the URN definitions and add fields for chartURL/dashboardURL into info aspects

* Rebase and address some comments
2020-11-12 11:17:22 -08:00
Nagarjuna Kanamarlapudi
7d574d1094
feat(field-level-lineage): Add models for field level lineage (#1936)
* feat(field-level-lineage): adding models for field level lineage

adding models for field level lineage. Introduce DatasetFieldUrn as a unique identifier for dataset field
2020-11-09 14:08:48 -08:00
Mars Lan
609739179b
refactor: drop unused models to prevent drifts (#1964) 2020-10-27 10:02:21 -07:00
John Plaisted
5e70f3648c Fix build after sync.
- Add build files for new module.
- Correctly edit TestUtils since it isn't synced.
- Reference new test utils.
- Delete duplicate pdl files.

SYNC=metadata-models_100.0.1
2020-09-24 16:02:12 -07:00
Kerem Sahin
7c7de50be8
feat(dashboard): Dashboard metadata models (#1775)
* Dashboard metadata models

* Update pairings in Contains and DownstreamOf relationships

* Adding ChartDataSourceType

* Extend chart types

* Make chart query an aspect & add chartIds to dashboard search document

* Add urn validation for chart & dashboard urns
2020-09-14 12:01:54 -07:00
John Plaisted
8223cdcbdb Fix build after merge:
- Add commonsLang to build file.
- Add emails field to CorpUserInfoDocument (either this should be synced or the index builder not synced in the future).
- Fix EbeanLocalDAOTest which used internal Urn API.
- Fix BaseSearchableEntityResource "backfill" override return types (and regenerate snapshots).
- EbeanlocalDAO's constructor changed; now requires URN class.
- Add restli resource module as dependency of :gms:api as it now contains a needed PDL model.
2020-09-11 09:15:56 -07:00
Arun Vasudevan
66dd008e3d
feat: add ML models (#1721)
* ML Model Schema Initial Version for feedback

* Added Deprecation Model

* Remove lock files

* Committing yarn lock file

* Fix Review Comments

* Using Common VersionTag Entity

* PR Review Comments Round-2

* Updated all model and feature references to MLModel and MLFeature

* Addressing PR Comments (Round 3)

* Updating Hyperparameter to a Map type

* Update to Dataset

* Review comments based on RFC

* ML Model Schema Initial Version for feedback

* Added Deprecation Model

* Remove lock files

* Committing yarn lock file

* Fix Review Comments

* Using Common VersionTag Entity

* PR Review Comments Round-2

* Updated all model and feature references to MLModel and MLFeature

* Addressing PR Comments (Round 3)

* Updating Hyperparameter to a Map type

* Update to Dataset

* fix: modify the etl script dependency (#1726)

Co-authored-by: Cobolbaby <Zhang.Xing-Long@inventec.com>

* fix: correct the way to catch the exception (#1727)

* fix: modify the etl script dependency

* fix: Correct the way to catch the exception

* fix: Compatible with the following kafka cluster when the Kafka Topic message Key cannot be empty

* fix: Adjust the kafka message key; Improve the comment of field

* fix: Avro schema required for key

Co-authored-by: Cobolbaby <Zhang.Xing-Long@inventec.com>

* refactor(models): remove internal cluster model (#1733)

* refactor(models): remove internal cluster model

Remove internal model which is not used in open source

* build(deps): bump lodash from 4.17.15 to 4.17.19 in /datahub-web (#1738)

Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update README.md

* Update README.md

* Update README.md

* Update the roadmap (#1740)

* Update the roadmap

- Make short term more like what we're doing this quarter
- Medium term is next quarter
- Long term is 2 or 3 quarters from now
- Visionary is even beyond that

Making this PR mostly to discuss the roadmap. I've moved a few items down to "unprioritized"; before merging this we should put these in a category. Mostly saving the state of what I've done so far.

* Update roadmap.md

Co-authored-by: Mars Lan <mars.th.lan@gmail.com>

* Update roadmap.md

* Update README.md

* doc: add a separate doc to keep track of the full list or links (#1744)

* Update README.md

* Create links.md

* Update README.md

* Update links.md

* Update README.md

* Update README.md

* Update features.md

* Update faq.md

* Update README.md

* Update README.md

* feat(gms): add postgres & mariadb supports to GMS (#1742)

* feat(gms): add postgres & mariadb supports to GMS

Also add corresponding docker-compose files

* Update README.md

* build(frontend): Drop unnecessary DB-related dependencies (#1741)

* refactor(frontend): Drop unnecessary DB-related dependencies

* Drop unused dependencies from top-level build script

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update links.md

* Update README.md

* Doc fixes

* Update roadmap.md

* Update faq.md

* Set theme jekyll-theme-cayman

* Create _config.yml

* Delete _config.yml

* Set theme jekyll-theme-cayman

* Update _config.yml

* Update _config.yml

* build: build GitHub Page from /docs directory (#1750)

- Move top-level MD files to /docs and symlink them back
- Update all absolute links to files in /docs to relative links

* Revert "build: build GitHub Page from /docs directory (#1750)" (#1751)

This reverts commit b0f56de7a81b8bf921ff37cb81024692d1b9a8ce.

* build: build GitHub Pages from /docs directory (#1752)

- Move non-README top-level MD files to /docs
- Update all absolute links to files in /docs to relative links
- Add a placeholder front page for GitHub Pages

* Update README.md

* Update README.md

* Update README.md

* feat(kafka-config): Add ability to configure other Kafka props (#1745)

* Integarte spring-kafka & spring-boot for security props

- Upgrade spring-kafka to 2.1.14
- Use KafkaListener and KafkaTemplates to enable KafkaAutoConfiguration
- Integrates spring-boot's KafkaProperties into spring-kafka's config

* Cleanup imports

* Add DataHub kafka env vars

* Remove kafka-streams dependency

* Add KafkaProperties to gms; Add docs

* Add to Adoption

* Remove KAFKA_BOOTSTRAP_SERVER default

Co-authored-by: jsotelo <javier.sotelo@viasat.com>
Co-authored-by: Kerem Sahin <ksahin@linkedin.com>

* Agenda for next town hall

* Update townhalls.md

* Update README.md

* Update README.md

* Add documentation around the DataHub RFC process. (#1754)

Other repos have similar RFC processes (though they seem to have a separate repo for their RFC docs).

This provides a more structured way for contributors to make siginficant design contributions.

https://github.com/linkedin/datahub/issues/1692

* metadata-models 72.0.8 -> 80.0.0 (#1756)

* <refactor>[ingestions]: align the default kafka topics with PR #1756 (#1758)

* docs: add a sequence diagram and a description (#1757)

* add a sequence diagram and a description

* update descrpition based on feedback

* Update README.md

* Update README.md

Co-authored-by: Mars Lan <mars.th.lan@gmail.com>

* Update README.md

* Fix reflinks in PR template (#1764)

* Update kafka-config.md (#1763)

Fix name of spring-kafka property to pass SASL_JAAS config

* Update entity.md

* Update README.md

* Update faq.md

* Update townhalls.md

* Update README.md

* Update townhalls.md

* Update townhalls.md

* docs: move quickstart guide to a separate file under docs (#1765)

docs: move quickstart guide to a separate doc under docs directory

* Update slack.md

* Update README.md

* Update slack.md

* Update metadata-ingestion.md

* Add workflow to check build and tests on PRs + releases. (#1769)

PRs are setup to skip docs.

Also, only run docker actions on linkedin/datahub (i.e. disable on forks; makes forks nicer since you don't have failing actions).

* Update developers.md

* Update developers.md

* Update README.md

* fix(models): remove unused model (#1748)

* fix(models): remove unused model

Fixes https://github.com/linkedin/datahub/issues/1719

* Drop DeploymentInfo from Dataset's value model & rebuild snapshot

* Update README.md

* Add a separate page for previous townhalls

* Update for August invite; link to history

* Update README.md

* build: remove travis (we're using GitHub actions). (#1770)

Remove travis (we're using GitHub actions).

Also ignore markdown in our current workflows.

Also update the README.md badge.

* update townhall date

* Update README.md

* Update townhalls.md

* build(docker): build & publish GitHub Package (#1771)

* build(docker): build & publish docker images to GitHub Packages

Will kepp publishing to Docker Hub meanwhile until all Dockerfiles have been updated to point to GitHub.
Fixes https://github.com/linkedin/datahub/issues/1548

* Rebase & fix dockerfile locations

* Update README.md

* Fix README.md

* docs: add placeholders for advanced topics (#1780)

* Create high-cardinality.md

* Create pdl-best-practices

* Create partial-update.md

* Rename pdl-best-practices to pdl-best-practices.md

* Create entity-hierarchy.md

* docs: more placeholders for advance topics (#1781)

* Create aspect-versioning.md

* Create derived-aspects.md

* Create backfilling.md

* Update README.md

* Update aspect-versioning.md

* Update aspect.md

* Update README.md

* Update townhall-history.md

* Update townhall-history.md

* Update rfc.md

* refactor(docker): make docker files easier to use during development. (#1777)

* Make docker files easier to use during development.

During development it quite nice to have docker work with locally built code. This allows you to launch all services very quickly, with your changes, and optionally with debugging support.

Changes made to docker files:
- Removed all redundant docker-compose files. We now have 1 giant file, and smaller files to use as overrides.
- Remove redundant README files that provided little information.
- Rename docker/<dir> to match the service name in the docker-compose file for clarity.
- Move environment variables to .env files. We only provide dev / the default environment for quickstart.
- Add debug options to docker files using multistage build to build minimal images with the idea that built files will be mounted instead.
- Add a docker/dev.sh script + compose file to easily use the dev override images (separate tag; images never published; uses debug docker files; mounts binaries to image).
- Added docs/docker documentation for this.

* build: fix docker actions. (#1787)

* bug: Fix docker actions.

We renamed directories in docker/ which broke the actions.

Also try to refactor the action files a little so that we can run (but not publish) these images on pull requests that change the docker/ dir as an extra check. Note this only seems to be supported by the dockerhub plugin; the github plugin doesn't support this (so that will be an issue when we move to it only).

* Drop extra pipes

* Update README.md

* refactor: remove unused model (#1788)

* refactor: remove unused internal models (#1789)

* docs: create search-over-new-field.md (#1790)

Add a doc on searching over a new field

* Update search-onboarding.md

* add description field for dataset index mapping (#1791)

* docs: how to customize the search experience (#1795)

* add description field for dataset index mapping

* documentation on how to customize the search experience

* feat(ingest): add example crawler for MS SQL (#1803)

Also fix the incorrect assumption on column comments & add sample docker-compose file

* Add log documentation

we didn't end up mounting logs; docker desktop is a better experience

* Update townhall-history.md

* Update quickstart.md

* fix(search): clear description from dataset index when it's cleared (#1808)

Fixes https://github.com/linkedin/datahub/issues/1798

* Update README.md

* Revert "Update README.md"

This reverts commit 74a0d7b262a2ac22de9bc52974b721d580914ff0.

* Update README.md

* Update README.md

* Update high-cardinality.md

* Update README.md

* Update relationship.md

* Update high-cardinality.md

* Update metadata-models to head! (#1811)

metadata-models 80.0.0 -> 90.0.13:

   90.0.13: Roll forward: Fix the open source build by avoiding URN method that isn't part of the open source URN.
    90.0.2: Refactor listUrnsFromIndex method
    90.0.0: Start distinguishing between [] aspects vs null aspects input param
    89.0.4: Fix the open source build by avoiding URN method that isn't part of the open source URN.
    89.0.2: fix some test case name
    89.0.0: META-12686: Made the MXE_v5 topics become strictly ACL'ed to avoid the wildcard write ACL as "MetadataXEvent.+"
    88.0.6: change DAO to take Storage Config as input
    88.0.3: Add a comment on lack of avro generation for MXEv5 + add MXEv5 to the pegasus validation task.
   87.0.15: META-12651: Integrate the metadata-models-ext with metadata-models
   87.0.13: add StorageConfig to Local DAO
    87.0.3: Treat empty aspect vs optional aspect same until all clients are migrated
    87.0.2: Treat empty aspect vs optional aspect differently
    87.0.1: META-12533: Skip processing unregistered aspect specific MAE.
    83.0.6: action method to return list of urns from strong consistent index
    83.0.4: Change input param type for batch backfill
    83.0.3: Implement batch backfill
    83.0.1: Implement support for OR filter in browse query
   82.0.10: Throw UnsupportedOperationException for unsupported condition types in search filter
    82.0.6: Implement local secondary backfilling index as part of backfill method
    82.0.5: [strongly consistent index] implement getUrns method
    82.0.4: Add indexing urn fields to the local secondary index
    82.0.0: Render Delta fiels in the MCE_v5.
    81.0.1: Add pegasus to avro conversion for FMCE
    80.0.4: add get all support for BaseSingleAspectEntitySimpleKeyResource
    80.0.2: Add a BaseSearchWriterDAO with an ESBulkWriterDAO implementation.
    80.0.1: META-12254: Produce aspect specific MAE with always emit option
    80.0.0: Convert getNodesInTraversedPath to getSubgraph to return complete view of the subgraph (nodes+edges)

* Update townhalls.md

* Update townhalls.md

* fix: drop the commits badge as it's flakey

* Update README.md

* fix: update defaults of aspectNames params (#1815)

fix: Update defaults of aspectNames params.

The last PR to sync internal code broke the external GMS, as code was now expected aspectNames to be null rather than empty by default. This preventing me logging into DataHub as the corp user request would fail (it assumed I asked for no aspects rather than all aspects).

TESTED: Built locally, launched with docker/dev.sh (so used latest frontend, but whatever). Verified I can now log into DataHub, browse and search for datasets, and view my profile.

* Update README.md

* Update README.md

* feat(kubernetes): Improve the security of the kubernetes/helm charts (#1782)

* 1747 | remove obsolete yaml files

* 1747 | remove configmap and its hardcoded references

* 1747 | add missing input parameter of neo4j.host

* 1747 | remove obsolete secrets and parameterize the rest

* 1747 | auto-generate gms secret

* 1747 | remove fullName overrides

* 1747 | fix parameters in subchart's values.yaml

* 1747 | remove hardcoding from parameters for gms host and port

* 1747 | upgrade chart version

* 1747 | update helm docs

* 1747 | add extraEnv, extraVolume and extraMounts

* 1747 | Alters pull policy of images to 'always' for ldh

Co-authored-by: shakti-garg <shakti.garg@gmail.com>

* Update README.md

* feat(data-platforms): adding rest resource for /dataPlatforms and mid-tier support (#1817)

* feat(data-platforms): Adding rest resource for /dataPlatforms and mid-tier support

* Removed data platforms which are Linkedin internal

* docs: add NOTICE (#1810)

* Copy NOTICE from wherehows

Copies the file from the wherehows branch.

* Update notice.

* Update links.md

* Update links.md

* Update README.md

* feat(dashboards): RFC for dashboards (#1778)

* feature(dashboards): RFC for dashboards

* Change directory structure

* Create goals & non-goals sections

* Removing alternatives section

* Update README.md

* Update links.md

* Update townhalls.md

* Update notice to include embedded licenses

Also list apache projects specifically.

* feat(frontend): update datahub-web client UI code (#1806)

* Releases updated version of datahub-web client UI code

* Fix typo in yarn lock

* Change yarn lock to match yarn registry directories

* Previous commit missed some paths

* Even more changes to yarnlock missing in previous commit

* Include codegen file for typings

* Add files to get parity for datahub-web and current OS datahub-midtier

* Add in typo fix from previous commit - change to proper license

* Implement proper OS fix for person entity picture url

* Workarounds for open source DH issues

* Fixes institutional memory api and removes unopensourced tabs for datasets

* Fixes search dataset deprecation and user search issue as a result of changes

* Remove internal only options in the avatar menu

* Update search-over-new-field.md

* docs: add external link (#1828)

* Update README.md

* Update links.md

* Review comments based on RFC

Co-authored-by: cobolbaby <cobolbaby@qq.com>
Co-authored-by: Cobolbaby <Zhang.Xing-Long@inventec.com>
Co-authored-by: Harsh Shah <hrshah@linkedin.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mars Lan <mars.th.lan@gmail.com>
Co-authored-by: John Plaisted <jplaisted@linkedin.com>
Co-authored-by: Kerem Sahin <ksahin@linkedin.com>
Co-authored-by: Javier Sotelo <javier.a.sotelo@gmail.com>
Co-authored-by: jsotelo <javier.sotelo@viasat.com>
Co-authored-by: Jyoti Wadhwani <jywadhwani@linkedin.com>
Co-authored-by: Chris Lee <wlee@linkedin.com>
Co-authored-by: Liangjun Jiang <ljiang510@gmail.com>
Co-authored-by: shakti-garg-saxo <68685481+shakti-garg-saxo@users.noreply.github.com>
Co-authored-by: na zhang <nazhang@linkedin.com>
Co-authored-by: shakti-garg <shakti.garg@gmail.com>
Co-authored-by: Charlie Tran <catran@linkedin.com>
2020-09-10 15:52:50 -07:00
Mars Lan
7d6fde4f37
feat: add MCE ingestion support for CorpGroup (#1837)
* feat: add MCE ingestion support for CorpGroup

Also use consistent camel case for corp user URNs in bootstrap MCE data

Fixes https://github.com/linkedin/datahub/issues/1822
2020-08-31 10:08:58 -07:00
Mars Lan
352eb3fff0
fix(models): remove unused model (#1748)
* fix(models): remove unused model

Fixes https://github.com/linkedin/datahub/issues/1719

* Drop DeploymentInfo from Dataset's value model & rebuild snapshot
2020-08-02 14:06:20 -07:00
Jyoti Wadhwani
779eaeed70
metadata-models 72.0.8 -> 80.0.0 (#1756) 2020-07-29 11:42:35 -07:00
Harsh Shah
136fc2a569
refactor(models): remove internal cluster model (#1733)
* refactor(models): remove internal cluster model

Remove internal model which is not used in open source
2020-07-15 16:32:45 -07:00
Liangjun Jiang
5d078aa617
Implemented data process search feature (#1706)
* implement search feature

* add test for dataprocessIndexBuilder; refactor code based on feedback

* update based on PR feedback

* Update DataProcessDocument.pdl

fixed typo wording.

* add not null check for data process info
2020-06-29 10:20:22 -07:00
Liangjun Jiang
40f08ecaf1
Implement data process graph feature (#1695)
* implement data process graph feature; update the document

* add unit test for data process graph

* removed auto generated avro files

* update image
2020-06-17 11:58:42 -07:00
Liangjun Jiang
92c4a3689e
Data process entity (#1680)
* add job info as aspect of a dataset

* add job urn def., aspect and entity

* job entity with upstream and downstream lineage

* use job urn in upstream & downstream

* add Job entity rest APIs

* rest.li api, impl and factory for job entity

* code cleanup

* use pdl; onboard data process entity

* add es index json

* fix gradlew build ignored tasks

* add a comment about data process info field

* fix style warning issues

* update content based on PR

* checked in generated snapshot json

* updated based on PR feedback

* update data process data format

* updated based on code review feedback

* revert back gms & mce-job docker image

* delete temp files

* update based pr feedback

* file name and a typo

* format with linkedin style

Co-authored-by: Liangjun <liajiang@expediagroup.com>
2020-06-09 15:42:08 -07:00
Mars Lan
1283dd3ff4
refactor(pdl): convert all pdsc to pdl (#1678)
Use the automated tool in https://linkedin.github.io/rest.li/pdl_migration
Also update all relevant docs
2020-05-21 10:49:23 -07:00
Mars Lan
36beab7da6
refactor: move rest.li keys from metadata-models to gms where they belong (#1617) 2020-04-01 11:02:39 -07:00
Kerem Sahin
1168501083 Enable tests for all modules by using global gradle config 2020-02-21 11:53:45 -08:00
Kerem Sahin
c8576ba503 Add some debuggin help & update default profile image link 2020-01-27 15:59:34 -08:00
Kerem Sahin
07a6e8b085 Remove dataset groups entity 2019-12-13 15:12:50 -08:00
Kerem Sahin
1a16f5fb2a Remove unused dataset models 2019-12-02 12:15:17 -08:00
Kerem Sahin
e2ad0f2adf corp-identity-gms 1.0.26 -> 1.0.40:
1.0.34: Downrank inactive users in user search query
    1.0.33: Refactor clients to remove snapshot builder
    1.0.32: Adding client & integration test for get_all
    1.0.30: Implement other clients for corp groups
    1.0.28: Add resources for search and autocomplete for corp groups
    1.0.27: Start using BaseClient from metadata-models
    1.0.26: Add get_all resource for CorpUsers

metadata-models 38.1.12 -> 50.0.6:
    50.0.2: Fix removed field update logic for all entities
    49.0.1: Add dataset graph builder with DownstreamOf relationship
    48.0.3: support query dao with traverse paths
    47.0.2: refactor the query dao with relationship filter model
    47.0.1: Fix for creating duplicate nodes when label for the node is missing
   46.0.21: extend filter model with relationship direction
   46.0.19: add unit test for entities partial update
   46.0.16: Allow relationship filter in the model and query dao
   46.0.15: support relationship directions for multi hop query
   46.0.14: Implementing reportsto relationship builder and corpuser graph builder
   46.0.10: refactor query dao interface using nullable to replace optional
    46.0.9: Rename Mock Utils to Test Utils in Metadata-models mp
    46.0.6: Remove search index config from metadata models
    46.0.2: neo4j query DAO with relationships directions support
    45.1.7: refactoring the graph relationship builders
    45.1.5: Use correct total count in search response
    45.1.3: Fix issue with empty search query filter
    45.1.2: Fix a bug with autocomplete limit param
    45.0.3: Change platform field type in the dataset search document
    45.0.2: implement multi hops query DAO with interface 5
    45.0.1: Moving dataset browsePaths build logic from wherehows-samza
    44.0.2: implement interface 2 in query DAO
    40.0.2: Only return records which exist in the DB after getting search hits
    39.0.0: Add a getAuditor method to BaseSnapshotResource rather than taking it in as a constructor argument
   38.1.13: Move BaseClient to metadata-models out of GMS template
   38.1.12: Remove default filtering on removed field for get_all

MP_VERSION=corp-identity-gms:1.0.40
MP_VERSION=metadata-models:50.0.6
MP_VERSION=wherehows-samza:1.0.56

This commit is automatically generated by li-opensource tool.
2019-11-19 02:27:28 -08:00
Kerem Sahin
3b27762d12 Remove unused data models 2019-10-08 18:28:24 -07:00
Kerem Sahin
d4df2b3e12 Removed unused data models 2019-10-05 15:20:58 -07:00
Kerem Sahin
f29a88c365 metadata-models 38.1.6 -> 38.1.8:
38.1.8: Add getFilter method as a search util
    38.1.7: Add index builder for corp groups

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.8
MP_VERSION=wherehows-samza:1.0.30

This commit is automaticaly generated by li-opensource tool.
2019-10-02 18:04:53 -07:00
Kerem Sahin
5bf797b216 corp-identity-gms 1.0.0 -> 1.0.25:
1.0.24: Corp user search across teams and skills
    1.0.21: Make /corpGroups /gridUsers /gridGroups extend BaseEntityResource
    1.0.17: Use correct util function to load resource file
    1.0.16: Add ingest, backfill & getSnapshot action methods to all top-level resources in corp-identity-gms
    1.0.13: Onboard search query templates on corp-identity-gms
     1.0.9: Fix batch get and add client for batch get
     1.0.8: Change package name for corpuser search config
     1.0.7: Use search config to get autocomplete field
     1.0.6: Implement searchable client
     1.0.5: Auto-complete backend support
     1.0.3: Add search API

metadata-models 24.0.0 -> 38.1.6:
    38.1.3: Index active status to corp user search index
    38.1.2: Change update response to create response for create API
   38.0.10: Mark BaseAspectResource as deprecated
    38.0.9: Allow TYPEREF items which have primitive types for arrays in models
    38.0.7: Add get-and-set-if-absent function to Local DAO
    38.0.6: Add find entities with one relationship in query dao
    38.0.4: Fix the inconsistency use of constants and urn params in query dao
    38.0.2: Parse source map to obtain the urn
    38.0.1: Search document validator in Index Builder
    38.0.0: Add urns to search result metadata
    37.0.7: Refactor the query dao
    37.0.6: Use test models in neo4j dao
    37.0.5: Drop metadata model structural assumptions made in neo4j DAOs
    37.0.2: Return empty list from getBrowsePaths if browsePaths field doesn't exist
    36.0.3: Drop elasticsearch-dao's metadata-models dependency
   35.0.10: ES Search DAO to handle null values
    35.0.5: Ebean local Dao query string match
    35.0.4: Drop all search & browse configs that have been moved to individual GMS
    35.0.3: Add ReportTo relationship model
    35.0.0: Load resource file properly
    34.0.9: Make RestliAuditor injectable
    34.0.8: Use encoded query in the test resource
    34.0.4: Handle empty aspects param correctly for backfill & getSnapshot actions
    34.0.1: Remove corp user specific files from metadata-models
    34.0.0: Add backfill & getSnapshot actions to BaseEntityResource
   32.0.16: Merge data template classes into metadata-dao's main artifact
   32.0.14: Replace "update" method with "ingest" action in BaseEntityResource
   32.0.13: Make fliter & sortCriteria parameters optional as they should have been
   32.0.12: Move AspectVersion & SnapshotKey back to their original namespaces
   32.0.11: Break metadata-dao's dependency on metadata-models
   32.0.10: Move model validators to a separate module
    32.0.9: Extract principal from the request context for user AuditStamp
    32.0.8: Fixing nullability annotations for search/autocomplete/browse resources & daos
    32.0.7: Move DAO-specific models to metadata-dao module
    32.0.4: Fix search finder not returning total search results count
    32.0.3: Implement get_all using search index
    32.0.2: Add missing nullability annotation
    32.0.1: Use more consistent naming for the test models
    31.0.1: Use test-specific metadata models in metadata-dao
    31.0.0: Add sort order to Search Dao
    30.0.2: Rename Aspect test model to AspectUnion to avoid confusion
    30.0.1: Committing migration for metadata-models.
    30.0.0: Add default autocomplete field in search config
   29.0.16: Modify testcase to account for empty filters
   29.0.15: Add searchable interface that clients can use
   29.0.14: Use test-specific metadata models in ebean-dao
   29.0.12: Move TestUtils to metadata-test-models module
   29.0.11: Refactor all tests in metadata-restli to use test models
   29.0.10: Move li-metadata-test-utils, metadata-test-models, metadata-test-utils into a new metadata-testing directory to improve code organization.
            Note that this is a backward compatible change as this doesn't alter the produced artifacts.
    29.0.9: Move test-specific models to a stand-alone module
    29.0.2: Refactor the rest of validators
    28.0.3: Refactor validateSchema for aspect
    28.0.2: Implement searchDao for CorpUserInfo.
   27.0.16: Refactor for ModelValidation tests
   27.0.10: Add new relationship union to model utils
    27.0.9: Add plugin to rest client factory
    27.0.6: Add rest high level factory
    27.0.5: Fix a test bug when reviewing the code
    27.0.4: Add create via lambda API to BaseVersionedAspectResource
    27.0.2: Change return type of search finder to capture search result metadata in BaseSearchableEntityResource
    27.0.1: Drop the unnecessary get method from BaseEntityResource
    27.0.0: Add BaseBrowsableEntityResource
   26.0.15: Add autocomplete action to BaseSearchableEntityResource
   26.0.14: Add BaseSearchableEntityResource
   26.0.13: Add getUrnFromDocument & urnClassForDocument util methods that are needed in future RBs
   26.0.11: Add BaseVersionedAspectResource
    26.0.9: Index signals associated with dataset relevance
    26.0.4: Support namespace for ID generation
    26.0.1: Fix inconsistent instance variable naming in SearchResult
    25.0.6: Add entity-snapshot conversion
    25.0.5: Use test-specific metadata models in metadata-restli
    25.0.3: Add aspect filtering to BaseEntityResource
    24.0.9: Add update method to BaseEntityResource
    24.0.7: Fix for parameter types of getBrowsePaths action method

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.6
MP_VERSION=wherehows-samza:1.0.29
2019-10-02 11:13:44 -07:00
Kerem Sahin
53b79de9bf Fix the issue with validating array types 2019-09-26 20:52:30 -07:00
Kerem Sahin
1ec31bbfb7 Start indexing upstream datasets to search index to support downstream dataset relationships 2019-09-26 11:57:58 -07:00
Seyi Adebajo
c57d7719a6
Updates default pictureLink in CorpUserEditableInfo
Updated to path https://raw.githubusercontent.com/linkedin/WhereHows/datahub/datahub-web/packages/data-portal/public/assets/images/default_avatar.png
2019-09-17 08:50:47 -07:00
Kerem Sahin
339c9bea86 Update default pictureLink 2019-09-16 15:55:11 -07:00
Chris Lee
fd7350decc Resolved union type in avro serialization. 2019-09-05 14:27:59 -07:00
Kerem Sahin
047157e3fe Remove classes/files which are not used 2019-09-01 15:41:24 -07:00
Kerem Sahin
9fbad946ef Add missing MXE models and fix .gitignore 2019-09-01 15:23:39 -07:00
Kerem Sahin
23339df23a Initial commit for Data Hub 2019-08-31 20:51:14 -07:00