12076 Commits

Author SHA1 Message Date
Kerem Sahin
beabee8ad8 Adding missing CorpGroupDaoFactory 2019-10-02 15:42:02 -07:00
Kerem Sahin
3d47d2bc2f Remove not used metadata-test-utils module 2019-10-02 11:54:15 -07:00
Kerem Sahin
5bf797b216 corp-identity-gms 1.0.0 -> 1.0.25:
1.0.24: Corp user search across teams and skills
    1.0.21: Make /corpGroups /gridUsers /gridGroups extend BaseEntityResource
    1.0.17: Use correct util function to load resource file
    1.0.16: Add ingest, backfill & getSnapshot action methods to all top-level resources in corp-identity-gms
    1.0.13: Onboard search query templates on corp-identity-gms
     1.0.9: Fix batch get and add client for batch get
     1.0.8: Change package name for corpuser search config
     1.0.7: Use search config to get autocomplete field
     1.0.6: Implement searchable client
     1.0.5: Auto-complete backend support
     1.0.3: Add search API

metadata-models 24.0.0 -> 38.1.6:
    38.1.3: Index active status to corp user search index
    38.1.2: Change update response to create response for create API
   38.0.10: Mark BaseAspectResource as deprecated
    38.0.9: Allow TYPEREF items which have primitive types for arrays in models
    38.0.7: Add get-and-set-if-absent function to Local DAO
    38.0.6: Add find entities with one relationship in query dao
    38.0.4: Fix the inconsistency use of constants and urn params in query dao
    38.0.2: Parse source map to obtain the urn
    38.0.1: Search document validator in Index Builder
    38.0.0: Add urns to search result metadata
    37.0.7: Refactor the query dao
    37.0.6: Use test models in neo4j dao
    37.0.5: Drop metadata model structural assumptions made in neo4j DAOs
    37.0.2: Return empty list from getBrowsePaths if browsePaths field doesn't exist
    36.0.3: Drop elasticsearch-dao's metadata-models dependency
   35.0.10: ES Search DAO to handle null values
    35.0.5: Ebean local Dao query string match
    35.0.4: Drop all search & browse configs that have been moved to individual GMS
    35.0.3: Add ReportTo relationship model
    35.0.0: Load resource file properly
    34.0.9: Make RestliAuditor injectable
    34.0.8: Use encoded query in the test resource
    34.0.4: Handle empty aspects param correctly for backfill & getSnapshot actions
    34.0.1: Remove corp user specific files from metadata-models
    34.0.0: Add backfill & getSnapshot actions to BaseEntityResource
   32.0.16: Merge data template classes into metadata-dao's main artifact
   32.0.14: Replace "update" method with "ingest" action in BaseEntityResource
   32.0.13: Make fliter & sortCriteria parameters optional as they should have been
   32.0.12: Move AspectVersion & SnapshotKey back to their original namespaces
   32.0.11: Break metadata-dao's dependency on metadata-models
   32.0.10: Move model validators to a separate module
    32.0.9: Extract principal from the request context for user AuditStamp
    32.0.8: Fixing nullability annotations for search/autocomplete/browse resources & daos
    32.0.7: Move DAO-specific models to metadata-dao module
    32.0.4: Fix search finder not returning total search results count
    32.0.3: Implement get_all using search index
    32.0.2: Add missing nullability annotation
    32.0.1: Use more consistent naming for the test models
    31.0.1: Use test-specific metadata models in metadata-dao
    31.0.0: Add sort order to Search Dao
    30.0.2: Rename Aspect test model to AspectUnion to avoid confusion
    30.0.1: Committing migration for metadata-models.
    30.0.0: Add default autocomplete field in search config
   29.0.16: Modify testcase to account for empty filters
   29.0.15: Add searchable interface that clients can use
   29.0.14: Use test-specific metadata models in ebean-dao
   29.0.12: Move TestUtils to metadata-test-models module
   29.0.11: Refactor all tests in metadata-restli to use test models
   29.0.10: Move li-metadata-test-utils, metadata-test-models, metadata-test-utils into a new metadata-testing directory to improve code organization.
            Note that this is a backward compatible change as this doesn't alter the produced artifacts.
    29.0.9: Move test-specific models to a stand-alone module
    29.0.2: Refactor the rest of validators
    28.0.3: Refactor validateSchema for aspect
    28.0.2: Implement searchDao for CorpUserInfo.
   27.0.16: Refactor for ModelValidation tests
   27.0.10: Add new relationship union to model utils
    27.0.9: Add plugin to rest client factory
    27.0.6: Add rest high level factory
    27.0.5: Fix a test bug when reviewing the code
    27.0.4: Add create via lambda API to BaseVersionedAspectResource
    27.0.2: Change return type of search finder to capture search result metadata in BaseSearchableEntityResource
    27.0.1: Drop the unnecessary get method from BaseEntityResource
    27.0.0: Add BaseBrowsableEntityResource
   26.0.15: Add autocomplete action to BaseSearchableEntityResource
   26.0.14: Add BaseSearchableEntityResource
   26.0.13: Add getUrnFromDocument & urnClassForDocument util methods that are needed in future RBs
   26.0.11: Add BaseVersionedAspectResource
    26.0.9: Index signals associated with dataset relevance
    26.0.4: Support namespace for ID generation
    26.0.1: Fix inconsistent instance variable naming in SearchResult
    25.0.6: Add entity-snapshot conversion
    25.0.5: Use test-specific metadata models in metadata-restli
    25.0.3: Add aspect filtering to BaseEntityResource
    24.0.9: Add update method to BaseEntityResource
    24.0.7: Fix for parameter types of getBrowsePaths action method

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.6
MP_VERSION=wherehows-samza:1.0.29
2019-10-02 11:13:44 -07:00
Kerem Sahin
234b2c2696 Update urn classes by adding deserialize methods 2019-10-01 14:54:24 -07:00
Kerem Sahin
f1b8392e1a Update README 2019-09-30 17:11:26 -07:00
Kerem Sahin
8928e7a147 Add more samples to bootstrap data to be able to show upstreams/downstreams support v0.1.1-alpha 2019-09-27 11:47:59 -07:00
Kerem Sahin
1fdd1e3cc3 Update GMS readme by adding sample curl calls for upstream&downstream relationships 2019-09-26 20:54:35 -07:00
Kerem Sahin
ea15628912 Add upstreams field to dataset search index mapping 2019-09-26 20:54:03 -07:00
Kerem Sahin
06fae51ca3 Fix NPE issue with downstream lineage resource 2019-09-26 20:53:38 -07:00
Kerem Sahin
53b79de9bf Fix the issue with validating array types 2019-09-26 20:52:30 -07:00
Kerem Sahin
40478e2d26 Adding support for surfacing downstream dataset relationships using search index until we have graph ready 2019-09-26 11:59:11 -07:00
Kerem Sahin
1ec31bbfb7 Start indexing upstream datasets to search index to support downstream dataset relationships 2019-09-26 11:57:58 -07:00
Chris Lee
646630b599 Enabled the kafka ingestion pipeline. 2019-09-24 16:41:06 -07:00
Chris Lee
49534ea590 Aligned the long type in time format. v0.1.0-alpha 2019-09-19 09:48:43 -07:00
Seyi Adebajo
6e158b2fd5
Merge pull request #1502 from theseyi/datahub
Fixes login tests related to token removal
2019-09-18 17:22:24 -07:00
Seyi Adebajo
59ed313bcd Fixes login tests related to token removal 2019-09-18 17:19:59 -07:00
Seyi Adebajo
81e34e8d22
Merge pull request #1501 from theseyi/datahub
Sets flag to show token credential input to false
2019-09-18 12:56:31 -07:00
Seyi Adebajo
6080ddfae9 Sets flag to show token credential input to false 2019-09-18 12:55:59 -07:00
Seyi Adebajo
dfd69b0f8a
Merge pull request #1500 from theseyi/datahub
Updates the tests and autocomplete facets to origin string
2019-09-18 12:46:09 -07:00
Seyi Adebajo
829a10ce09 Updates the tests and autocomplete facets to origin string 2019-09-18 12:45:31 -07:00
Seyi Adebajo
33a493e0c3
Merge pull request #1499 from theseyi/datahub
Updates the origin facet to origin instead of datatorigin
2019-09-18 12:14:28 -07:00
Seyi Adebajo
424e7f79cb Updates the origin facet to origin instead of datatorigin 2019-09-18 11:56:22 -07:00
Seyi Adebajo
e482add6b2
Merge pull request #1498 from theseyi/datahub
Resolves related ticket issues and misc tasks readying for OS
2019-09-18 11:19:34 -07:00
Chris Lee
600c2f04ab Organized the output layout. 2019-09-18 11:03:48 -07:00
Seyi Adebajo
f66f81dc4c META-9929, META-9868, resolves related ticket issues and misc tasks 2019-09-18 10:07:04 -07:00
Kerem Sahin
162d52a421 Updated DataHub wiki link conf 2019-09-17 16:30:37 -07:00
Chris Lee
72b18aa890 Introduced the hive ETL pipeline. 2019-09-17 16:14:09 -07:00
Chris Lee
57976cc901 Introduced the hive ETL pipeline. 2019-09-17 16:13:35 -07:00
Chris Lee
9cc7a1e2f8 Imported the confluent-kafka[avro] as a requirement. 2019-09-17 15:06:56 -07:00
Chris Lee
e6b98c1ba1 Merge branch 'datahub' of https://github.com/linkedin/WhereHows into datahub 2019-09-17 15:04:02 -07:00
Chris Lee
5c2e03b162 Imported the confluent-kafka[avro] as a requirement. 2019-09-17 15:03:25 -07:00
Seyi Adebajo
c57d7719a6
Updates default pictureLink in CorpUserEditableInfo
Updated to path https://raw.githubusercontent.com/linkedin/WhereHows/datahub/datahub-web/packages/data-portal/public/assets/images/default_avatar.png
2019-09-17 08:50:47 -07:00
Kerem Sahin
339c9bea86 Update default pictureLink 2019-09-16 15:55:11 -07:00
Chris Lee
493cbe16d6 Added the runbook for the requirements.txt. 2019-09-16 13:27:16 -07:00
Chris Lee
bd32ec20a1 Supplied the requirements.txt for metadata-ingestion. 2019-09-16 13:13:42 -07:00
Chris Lee
81a38f87f7 Classified the ETL jobs in metadata-ingestion. 2019-09-16 12:52:37 -07:00
Kerem Sahin
28b876f323 Update docs: No need for running initialization script for creating search indices 2019-09-12 19:06:35 -07:00
Kerem Sahin
1a3ddff4a4 Creating an image for elasticsearch-setup to automatically create indices 2019-09-12 19:04:29 -07:00
Chris Lee
430f13108d Enabled the configuration lists in the LDAP ETL. 2019-09-12 10:16:41 -07:00
Chris Lee
9df9d336c9 Ingested metadata from LDAP server to Data Hub. 2019-09-10 18:58:22 -07:00
Kerem Sahin
1bf0ff72b4 Doc update: adding docker-compose pull before up to always get the latest version of images 2019-09-10 17:51:17 -07:00
Kerem Sahin
c365c406e1 Normalize browse paths for datasets 2019-09-10 15:14:40 -07:00
Kerem Sahin
6716e38279 Update documentation 2019-09-09 01:45:49 -07:00
Kerem Sahin
8bfb086e09 Set displayName fields for corp user MCEs 2019-09-09 00:31:09 -07:00
Kerem Sahin
822dc22d8e Add pictureLink to DatasetOwner view 2019-09-08 23:59:25 -07:00
Kerem Sahin
fcd1b46f41 Add dataset urn to dataset client API value model 2019-09-08 23:02:44 -07:00
Kerem Sahin
0dc5bd9fe0 Add documentation 2019-09-08 20:25:58 -07:00
Kerem Sahin
693ee97c6c Fix ingestion script by pointing to correct MCE schema
Refactor for metadata-ingestion module
Adding readme for metadata-ingestion
2019-09-08 05:27:19 -07:00
Kerem Sahin
e3e6dc1d30 Remove kafka rest-proxy container which is not required 2019-09-08 01:10:38 -07:00
Kerem Sahin
9f43305fe9 Add mce-consumer to quickstart docker-compose 2019-09-08 00:57:04 -07:00