12004 Commits

Author SHA1 Message Date
Chris Lee
89c6e35502 Performed --user flag to avoid the permission issue on requirement installation. 2019-11-13 17:01:32 -08:00
Kerem Sahin
d8dfa2ddf0
Add install requirements step in the quickstart section 2019-11-13 15:10:22 -08:00
Kerem Sahin
bae624eaf3 Move init commands out of Dockerfile and put it inside docker-compose.yml and increase timeout for datahub-gms to 240s 2019-11-12 19:13:56 -08:00
Kerem Sahin
4500e9ce7b Set Elasticsearch Docker container max heap size to 1GB and increase timeout to 120s for Elasticsearch to be ready 2019-11-12 18:13:37 -08:00
Kerem Sahin
8ff0b954ca
Update quickstart section in README.md 2019-11-11 12:38:05 -08:00
Chris Lee
1d6484da60 Released mySQL ETL in metadata-ingestion. 2019-10-24 05:17:28 -07:00
Mars Lan
5193304678 Update build.gradle 2019-10-22 10:45:14 -07:00
Mars Lan
8956ed89c4 Set Java 8 compatibility in gradle 2019-10-22 10:45:14 -07:00
Mars Lan
3a57d324ad
Update README.md 2019-10-21 05:47:17 -07:00
Seyi Adebajo
46c55c2635
Merge pull request #1506 from theseyi/datahub
Remove local facet construction for platform and origin autocomplete …
2019-10-16 20:03:51 -07:00
Seyi Adebajo
1119f4b28b Remove local facet construction for platform and origin autocomplete facets, update tests for autocomplete platform and origin 2019-10-16 16:15:59 -07:00
Kerem Sahin
3b27762d12 Remove unused data models 2019-10-08 18:28:24 -07:00
Seyi Adebajo
f3727edaac
Merge pull request #1503 from theseyi/datahub
Reverts accidental deletion of tests/acceptance/login-test.js, refact…
2019-10-07 16:05:40 -07:00
Kerem Sahin
77bb9b93db Update travis configuration to optimize build time 2019-10-07 15:38:12 -07:00
Seyi Adebajo
8f1f786e2a Miscellaneous fixes from last meeting on OSS 2019-10-07 12:08:35 -07:00
Seyi Adebajo
95cc71d407 Reverts accidental deletion of tests/acceptance/login-test.js, refactors and converts to ts 2019-10-07 09:39:23 -07:00
Kerem Sahin
bb38a9467a Add --build flag when using docker-compose up to always build elasticsearch-setup image 2019-10-05 15:28:54 -07:00
Kerem Sahin
b48ea27c26 Bump docker image versions to latest in quickstart after fixing issues 2019-10-05 15:28:12 -07:00
Kerem Sahin
d4df2b3e12 Removed unused data models 2019-10-05 15:20:58 -07:00
Kerem Sahin
d3e2ab2bf8 Remove unused search templates & configs 2019-10-04 19:28:11 -07:00
Kerem Sahin
9878cfe632 corp-identity-gms 1.0.25 -> 1.0.26:
1.0.26: Add get_all resource for CorpUsers

metadata-models 38.1.8 -> 38.1.12:
   38.1.12: Remove default filtering on removed field for get_all
   38.1.11: Add index builders for datasets and corp groups
    38.1.8: Add getFilter method as a search util

MP_VERSION=corp-identity-gms:1.0.26
MP_VERSION=metadata-models:38.1.12
MP_VERSION=wherehows-samza:1.0.31
2019-10-04 19:22:34 -07:00
Kerem Sahin
878cb6dde2 Get overwritten changes back 2019-10-03 19:27:41 -07:00
Kerem Sahin
c65a65c0b7 Update corp user search index mapping 2019-10-03 19:26:28 -07:00
Kerem Sahin
54270f12f0 Rename serach query templates for dataset search 2019-10-03 19:25:57 -07:00
Kerem Sahin
201f2e0fb5 Changed dependency versions of Data Hub Docker images to release-0.1.1-alpha in quickstart docker-compose 2019-10-03 13:44:19 -07:00
Kerem Sahin
f29a88c365 metadata-models 38.1.6 -> 38.1.8:
38.1.8: Add getFilter method as a search util
    38.1.7: Add index builder for corp groups

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.8
MP_VERSION=wherehows-samza:1.0.30

This commit is automaticaly generated by li-opensource tool.
2019-10-02 18:04:53 -07:00
Kerem Sahin
8e809e383d Fix Kafka topic names 2019-10-02 16:28:03 -07:00
Kerem Sahin
9f4e6320c5 Fix case bug for CorpUser search bean 2019-10-02 15:56:37 -07:00
Kerem Sahin
beabee8ad8 Adding missing CorpGroupDaoFactory 2019-10-02 15:42:02 -07:00
Kerem Sahin
3d47d2bc2f Remove not used metadata-test-utils module 2019-10-02 11:54:15 -07:00
Kerem Sahin
5bf797b216 corp-identity-gms 1.0.0 -> 1.0.25:
1.0.24: Corp user search across teams and skills
    1.0.21: Make /corpGroups /gridUsers /gridGroups extend BaseEntityResource
    1.0.17: Use correct util function to load resource file
    1.0.16: Add ingest, backfill & getSnapshot action methods to all top-level resources in corp-identity-gms
    1.0.13: Onboard search query templates on corp-identity-gms
     1.0.9: Fix batch get and add client for batch get
     1.0.8: Change package name for corpuser search config
     1.0.7: Use search config to get autocomplete field
     1.0.6: Implement searchable client
     1.0.5: Auto-complete backend support
     1.0.3: Add search API

metadata-models 24.0.0 -> 38.1.6:
    38.1.3: Index active status to corp user search index
    38.1.2: Change update response to create response for create API
   38.0.10: Mark BaseAspectResource as deprecated
    38.0.9: Allow TYPEREF items which have primitive types for arrays in models
    38.0.7: Add get-and-set-if-absent function to Local DAO
    38.0.6: Add find entities with one relationship in query dao
    38.0.4: Fix the inconsistency use of constants and urn params in query dao
    38.0.2: Parse source map to obtain the urn
    38.0.1: Search document validator in Index Builder
    38.0.0: Add urns to search result metadata
    37.0.7: Refactor the query dao
    37.0.6: Use test models in neo4j dao
    37.0.5: Drop metadata model structural assumptions made in neo4j DAOs
    37.0.2: Return empty list from getBrowsePaths if browsePaths field doesn't exist
    36.0.3: Drop elasticsearch-dao's metadata-models dependency
   35.0.10: ES Search DAO to handle null values
    35.0.5: Ebean local Dao query string match
    35.0.4: Drop all search & browse configs that have been moved to individual GMS
    35.0.3: Add ReportTo relationship model
    35.0.0: Load resource file properly
    34.0.9: Make RestliAuditor injectable
    34.0.8: Use encoded query in the test resource
    34.0.4: Handle empty aspects param correctly for backfill & getSnapshot actions
    34.0.1: Remove corp user specific files from metadata-models
    34.0.0: Add backfill & getSnapshot actions to BaseEntityResource
   32.0.16: Merge data template classes into metadata-dao's main artifact
   32.0.14: Replace "update" method with "ingest" action in BaseEntityResource
   32.0.13: Make fliter & sortCriteria parameters optional as they should have been
   32.0.12: Move AspectVersion & SnapshotKey back to their original namespaces
   32.0.11: Break metadata-dao's dependency on metadata-models
   32.0.10: Move model validators to a separate module
    32.0.9: Extract principal from the request context for user AuditStamp
    32.0.8: Fixing nullability annotations for search/autocomplete/browse resources & daos
    32.0.7: Move DAO-specific models to metadata-dao module
    32.0.4: Fix search finder not returning total search results count
    32.0.3: Implement get_all using search index
    32.0.2: Add missing nullability annotation
    32.0.1: Use more consistent naming for the test models
    31.0.1: Use test-specific metadata models in metadata-dao
    31.0.0: Add sort order to Search Dao
    30.0.2: Rename Aspect test model to AspectUnion to avoid confusion
    30.0.1: Committing migration for metadata-models.
    30.0.0: Add default autocomplete field in search config
   29.0.16: Modify testcase to account for empty filters
   29.0.15: Add searchable interface that clients can use
   29.0.14: Use test-specific metadata models in ebean-dao
   29.0.12: Move TestUtils to metadata-test-models module
   29.0.11: Refactor all tests in metadata-restli to use test models
   29.0.10: Move li-metadata-test-utils, metadata-test-models, metadata-test-utils into a new metadata-testing directory to improve code organization.
            Note that this is a backward compatible change as this doesn't alter the produced artifacts.
    29.0.9: Move test-specific models to a stand-alone module
    29.0.2: Refactor the rest of validators
    28.0.3: Refactor validateSchema for aspect
    28.0.2: Implement searchDao for CorpUserInfo.
   27.0.16: Refactor for ModelValidation tests
   27.0.10: Add new relationship union to model utils
    27.0.9: Add plugin to rest client factory
    27.0.6: Add rest high level factory
    27.0.5: Fix a test bug when reviewing the code
    27.0.4: Add create via lambda API to BaseVersionedAspectResource
    27.0.2: Change return type of search finder to capture search result metadata in BaseSearchableEntityResource
    27.0.1: Drop the unnecessary get method from BaseEntityResource
    27.0.0: Add BaseBrowsableEntityResource
   26.0.15: Add autocomplete action to BaseSearchableEntityResource
   26.0.14: Add BaseSearchableEntityResource
   26.0.13: Add getUrnFromDocument & urnClassForDocument util methods that are needed in future RBs
   26.0.11: Add BaseVersionedAspectResource
    26.0.9: Index signals associated with dataset relevance
    26.0.4: Support namespace for ID generation
    26.0.1: Fix inconsistent instance variable naming in SearchResult
    25.0.6: Add entity-snapshot conversion
    25.0.5: Use test-specific metadata models in metadata-restli
    25.0.3: Add aspect filtering to BaseEntityResource
    24.0.9: Add update method to BaseEntityResource
    24.0.7: Fix for parameter types of getBrowsePaths action method

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.6
MP_VERSION=wherehows-samza:1.0.29
2019-10-02 11:13:44 -07:00
Kerem Sahin
234b2c2696 Update urn classes by adding deserialize methods 2019-10-01 14:54:24 -07:00
Kerem Sahin
f1b8392e1a Update README 2019-09-30 17:11:26 -07:00
Kerem Sahin
8928e7a147 Add more samples to bootstrap data to be able to show upstreams/downstreams support v0.1.1-alpha 2019-09-27 11:47:59 -07:00
Kerem Sahin
1fdd1e3cc3 Update GMS readme by adding sample curl calls for upstream&downstream relationships 2019-09-26 20:54:35 -07:00
Kerem Sahin
ea15628912 Add upstreams field to dataset search index mapping 2019-09-26 20:54:03 -07:00
Kerem Sahin
06fae51ca3 Fix NPE issue with downstream lineage resource 2019-09-26 20:53:38 -07:00
Kerem Sahin
53b79de9bf Fix the issue with validating array types 2019-09-26 20:52:30 -07:00
Kerem Sahin
40478e2d26 Adding support for surfacing downstream dataset relationships using search index until we have graph ready 2019-09-26 11:59:11 -07:00
Kerem Sahin
1ec31bbfb7 Start indexing upstream datasets to search index to support downstream dataset relationships 2019-09-26 11:57:58 -07:00
Chris Lee
646630b599 Enabled the kafka ingestion pipeline. 2019-09-24 16:41:06 -07:00
Chris Lee
49534ea590 Aligned the long type in time format. v0.1.0-alpha 2019-09-19 09:48:43 -07:00
Seyi Adebajo
6e158b2fd5
Merge pull request #1502 from theseyi/datahub
Fixes login tests related to token removal
2019-09-18 17:22:24 -07:00
Seyi Adebajo
59ed313bcd Fixes login tests related to token removal 2019-09-18 17:19:59 -07:00
Seyi Adebajo
81e34e8d22
Merge pull request #1501 from theseyi/datahub
Sets flag to show token credential input to false
2019-09-18 12:56:31 -07:00
Seyi Adebajo
6080ddfae9 Sets flag to show token credential input to false 2019-09-18 12:55:59 -07:00
Seyi Adebajo
dfd69b0f8a
Merge pull request #1500 from theseyi/datahub
Updates the tests and autocomplete facets to origin string
2019-09-18 12:46:09 -07:00
Seyi Adebajo
829a10ce09 Updates the tests and autocomplete facets to origin string 2019-09-18 12:45:31 -07:00
Seyi Adebajo
33a493e0c3
Merge pull request #1499 from theseyi/datahub
Updates the origin facet to origin instead of datatorigin
2019-09-18 12:14:28 -07:00
Seyi Adebajo
424e7f79cb Updates the origin facet to origin instead of datatorigin 2019-09-18 11:56:22 -07:00