292 Commits

Author SHA1 Message Date
Kerem Sahin
e905822034 Fix some changes which came with automatic commit 2019-11-19 03:08:00 -08:00
Kerem Sahin
e2ad0f2adf corp-identity-gms 1.0.26 -> 1.0.40:
1.0.34: Downrank inactive users in user search query
    1.0.33: Refactor clients to remove snapshot builder
    1.0.32: Adding client & integration test for get_all
    1.0.30: Implement other clients for corp groups
    1.0.28: Add resources for search and autocomplete for corp groups
    1.0.27: Start using BaseClient from metadata-models
    1.0.26: Add get_all resource for CorpUsers

metadata-models 38.1.12 -> 50.0.6:
    50.0.2: Fix removed field update logic for all entities
    49.0.1: Add dataset graph builder with DownstreamOf relationship
    48.0.3: support query dao with traverse paths
    47.0.2: refactor the query dao with relationship filter model
    47.0.1: Fix for creating duplicate nodes when label for the node is missing
   46.0.21: extend filter model with relationship direction
   46.0.19: add unit test for entities partial update
   46.0.16: Allow relationship filter in the model and query dao
   46.0.15: support relationship directions for multi hop query
   46.0.14: Implementing reportsto relationship builder and corpuser graph builder
   46.0.10: refactor query dao interface using nullable to replace optional
    46.0.9: Rename Mock Utils to Test Utils in Metadata-models mp
    46.0.6: Remove search index config from metadata models
    46.0.2: neo4j query DAO with relationships directions support
    45.1.7: refactoring the graph relationship builders
    45.1.5: Use correct total count in search response
    45.1.3: Fix issue with empty search query filter
    45.1.2: Fix a bug with autocomplete limit param
    45.0.3: Change platform field type in the dataset search document
    45.0.2: implement multi hops query DAO with interface 5
    45.0.1: Moving dataset browsePaths build logic from wherehows-samza
    44.0.2: implement interface 2 in query DAO
    40.0.2: Only return records which exist in the DB after getting search hits
    39.0.0: Add a getAuditor method to BaseSnapshotResource rather than taking it in as a constructor argument
   38.1.13: Move BaseClient to metadata-models out of GMS template
   38.1.12: Remove default filtering on removed field for get_all

MP_VERSION=corp-identity-gms:1.0.40
MP_VERSION=metadata-models:50.0.6
MP_VERSION=wherehows-samza:1.0.56

This commit is automatically generated by li-opensource tool.
2019-11-19 02:27:28 -08:00
Mars Lan
5193304678 Update build.gradle 2019-10-22 10:45:14 -07:00
Mars Lan
8956ed89c4 Set Java 8 compatibility in gradle 2019-10-22 10:45:14 -07:00
Kerem Sahin
5bf797b216 corp-identity-gms 1.0.0 -> 1.0.25:
1.0.24: Corp user search across teams and skills
    1.0.21: Make /corpGroups /gridUsers /gridGroups extend BaseEntityResource
    1.0.17: Use correct util function to load resource file
    1.0.16: Add ingest, backfill & getSnapshot action methods to all top-level resources in corp-identity-gms
    1.0.13: Onboard search query templates on corp-identity-gms
     1.0.9: Fix batch get and add client for batch get
     1.0.8: Change package name for corpuser search config
     1.0.7: Use search config to get autocomplete field
     1.0.6: Implement searchable client
     1.0.5: Auto-complete backend support
     1.0.3: Add search API

metadata-models 24.0.0 -> 38.1.6:
    38.1.3: Index active status to corp user search index
    38.1.2: Change update response to create response for create API
   38.0.10: Mark BaseAspectResource as deprecated
    38.0.9: Allow TYPEREF items which have primitive types for arrays in models
    38.0.7: Add get-and-set-if-absent function to Local DAO
    38.0.6: Add find entities with one relationship in query dao
    38.0.4: Fix the inconsistency use of constants and urn params in query dao
    38.0.2: Parse source map to obtain the urn
    38.0.1: Search document validator in Index Builder
    38.0.0: Add urns to search result metadata
    37.0.7: Refactor the query dao
    37.0.6: Use test models in neo4j dao
    37.0.5: Drop metadata model structural assumptions made in neo4j DAOs
    37.0.2: Return empty list from getBrowsePaths if browsePaths field doesn't exist
    36.0.3: Drop elasticsearch-dao's metadata-models dependency
   35.0.10: ES Search DAO to handle null values
    35.0.5: Ebean local Dao query string match
    35.0.4: Drop all search & browse configs that have been moved to individual GMS
    35.0.3: Add ReportTo relationship model
    35.0.0: Load resource file properly
    34.0.9: Make RestliAuditor injectable
    34.0.8: Use encoded query in the test resource
    34.0.4: Handle empty aspects param correctly for backfill & getSnapshot actions
    34.0.1: Remove corp user specific files from metadata-models
    34.0.0: Add backfill & getSnapshot actions to BaseEntityResource
   32.0.16: Merge data template classes into metadata-dao's main artifact
   32.0.14: Replace "update" method with "ingest" action in BaseEntityResource
   32.0.13: Make fliter & sortCriteria parameters optional as they should have been
   32.0.12: Move AspectVersion & SnapshotKey back to their original namespaces
   32.0.11: Break metadata-dao's dependency on metadata-models
   32.0.10: Move model validators to a separate module
    32.0.9: Extract principal from the request context for user AuditStamp
    32.0.8: Fixing nullability annotations for search/autocomplete/browse resources & daos
    32.0.7: Move DAO-specific models to metadata-dao module
    32.0.4: Fix search finder not returning total search results count
    32.0.3: Implement get_all using search index
    32.0.2: Add missing nullability annotation
    32.0.1: Use more consistent naming for the test models
    31.0.1: Use test-specific metadata models in metadata-dao
    31.0.0: Add sort order to Search Dao
    30.0.2: Rename Aspect test model to AspectUnion to avoid confusion
    30.0.1: Committing migration for metadata-models.
    30.0.0: Add default autocomplete field in search config
   29.0.16: Modify testcase to account for empty filters
   29.0.15: Add searchable interface that clients can use
   29.0.14: Use test-specific metadata models in ebean-dao
   29.0.12: Move TestUtils to metadata-test-models module
   29.0.11: Refactor all tests in metadata-restli to use test models
   29.0.10: Move li-metadata-test-utils, metadata-test-models, metadata-test-utils into a new metadata-testing directory to improve code organization.
            Note that this is a backward compatible change as this doesn't alter the produced artifacts.
    29.0.9: Move test-specific models to a stand-alone module
    29.0.2: Refactor the rest of validators
    28.0.3: Refactor validateSchema for aspect
    28.0.2: Implement searchDao for CorpUserInfo.
   27.0.16: Refactor for ModelValidation tests
   27.0.10: Add new relationship union to model utils
    27.0.9: Add plugin to rest client factory
    27.0.6: Add rest high level factory
    27.0.5: Fix a test bug when reviewing the code
    27.0.4: Add create via lambda API to BaseVersionedAspectResource
    27.0.2: Change return type of search finder to capture search result metadata in BaseSearchableEntityResource
    27.0.1: Drop the unnecessary get method from BaseEntityResource
    27.0.0: Add BaseBrowsableEntityResource
   26.0.15: Add autocomplete action to BaseSearchableEntityResource
   26.0.14: Add BaseSearchableEntityResource
   26.0.13: Add getUrnFromDocument & urnClassForDocument util methods that are needed in future RBs
   26.0.11: Add BaseVersionedAspectResource
    26.0.9: Index signals associated with dataset relevance
    26.0.4: Support namespace for ID generation
    26.0.1: Fix inconsistent instance variable naming in SearchResult
    25.0.6: Add entity-snapshot conversion
    25.0.5: Use test-specific metadata models in metadata-restli
    25.0.3: Add aspect filtering to BaseEntityResource
    24.0.9: Add update method to BaseEntityResource
    24.0.7: Fix for parameter types of getBrowsePaths action method

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.6
MP_VERSION=wherehows-samza:1.0.29
2019-10-02 11:13:44 -07:00
Kerem Sahin
cc93351b63 Bump kafka-clients & kafka-streams to 2.3.0 2019-09-07 23:35:23 -07:00
Kerem Sahin
3f92af3928 Bump spring dependency to 5.1.6 2019-09-04 20:19:45 -07:00
Kerem Sahin
a3c78b7311 Bump spring framework to 4.1.9 2019-09-03 12:24:08 -07:00
Kerem Sahin
23339df23a Initial commit for Data Hub 2019-08-31 20:51:14 -07:00
Mars Lan
d18595f39d
Update Gralde to 4.9 & Scala to 2.11 (#1315)
* Update Gralde to 4.9 & Scala to 2.11

* Update play.gradle

* Update repositories.gradle
2018-08-10 17:51:30 -07:00
ziveo
8655c407c9 Update build.gradle (#1202)
* Update build.gradle

Updating Gradle version to at the moment lates 4.8

* Updating gradle-wrapper.properties and gradle-wrapper.jar files
2018-06-08 07:33:23 -07:00
ziveo
3763f8c1c6 Update build.gradle (#1167)
Updating Gradle version to at the moment lates 4.7
2018-05-16 22:02:58 -07:00
Seyi Adebajo
05022f5f2c adds code coverage support to ci: uses code-cov for static analysis, adds forked version for e-c-code-coverage with support for transpiled sources. updates gradle build to run ember test with coverage. removes custom yarn version. removes broken fancy tree dep. minor refactor to WIP fetch wrapper: adds doc and exports 2017-11-27 10:44:00 -08:00
Mars Lan
9f7341f542 Clean up the log4j vs slf4j mess and consolidate the dependency resolution into a gradle script (#745) 2017-09-13 15:37:29 -07:00
Mars Lan
ac25412cb7 Move java plugin back to applicable subprojects, instead of applying it broadly (#646) 2017-08-08 17:12:57 -07:00
Mars Lan
d1e644a7ec Add gradle support for building and packaging the ember web app (#409)
* Add gradle support for building and packaging the ember web app, alleviating the manual instlalation of node/npm/bower/ember.

* Remove generated files under web/public

* Update ember.gradle
2017-07-10 09:55:15 -07:00
Mars Lan
1efa4eef16 Add hooks for LI-specific adaptation of the build script. 2017-07-10 09:54:09 -07:00
Mars Lan
a589abbd76 Split the root build script into multiple scripts. (#348)
Split the root build script into multiple scripts
Add coveralls support.
2017-07-10 09:54:08 -07:00
Mars Lan
5a999f29b1 Revert "Split the root build script into multiple scripts."
This reverts commit 4b8a6f86577739209b09ec8cc8cb09c2808f4aa7.
2017-07-10 09:54:08 -07:00
Mars Lan
edf5c54de3 Split the root build script into multiple scripts.
Add support for coveralls.
2017-07-10 09:54:08 -07:00
Mars Lan
e36a40cd65 Generate code coverage reports (#334)
* Add playCoverage task to run code coverage using JaCoco for backend and web.

* Add jacocoTestReport task to run code coverage for testNG-based tests in wherehows-common & metadata-etl.
2017-07-10 09:53:28 -07:00
Shuya Tsukamoto
e0f324c8e4 Implement ParquetFileAnalyzer for hadoop-dataset-extractor-standalone (#483)
* Implement ParquetFileAnalyzer for hadoop-dataset-extractor-standalone

* Move location

* Update build.gradle
2017-05-04 10:08:51 -07:00
Eric Sun
7b36d09b58 Add get_schema_literal_from_url() to fetch schema literal based on schema url (#268)
* use schema_url_helper to fetch avro schema from hdfs or http location

* trim space

* add dfs.namenode.kerberos.principal.pattern; include htrace for SchemaUrlHelper
2016-11-07 08:14:45 -08:00
Yi Wang
664e4072bb Upgrade to play 2.4.8 2016-10-19 17:42:28 -07:00
Yi Wang
5049c847fa Update Kafka consumer actors to reduce memory usage 2016-10-10 14:49:14 -07:00
Eric Sun
1cd5872369 temp fix for hdfs_schema_crawler getRuntime().exec() hangs problem; exclude log4j 2016-08-03 15:50:00 -07:00
Eric Sun
67479eac58 log_dataset_instance_load_status 2016-08-01 13:00:39 -07:00
Yi Wang
6d4706bc62 Ingest Gobblin tracking events into wherehows using Kafka consumer client 2016-07-25 15:03:29 -07:00
jbai
e8ee8897e4 change the akka dependency to version 2.2.5 2016-07-11 11:18:52 -07:00
jbai
442b39ca2f change the akka version to 2.3.5 to fix the no such method issue 2016-07-01 15:14:20 -07:00
Eric Sun
35fe1532be more excludes for [licenseFormatMain] task 2016-06-28 23:28:06 -07:00
Eric Sun
1573fdb212 rename hive dependency to hive_exec; reuse metadata-etl/extralibs; test travis ci; 2016-06-28 18:03:02 -07:00
Eric Sun
56cebd0d9c add dict_dataset_instance and cfg_object_name_map to track dataset on multiple clusters and their replication dependency and view dependency 2016-06-22 19:40:05 -07:00
Rafal Kluszczynski
b28f7d924f adding gradle wrapper instead of manually downloded gradle distribution 2016-05-16 14:58:28 +02:00
SunZhaonan
4b3b344b96 enable travis 2016-03-15 16:20:04 -07:00
SunZhaonan
b5d7c38b7d Eclipse integration. Resolve circular dependency. wherehows-common-test configure. 2016-02-09 15:50:49 -08:00
SunZhaonan
0eddd8accf close connection appropriatly, minior fix 2015-12-21 12:07:08 -08:00
SunZhaonan
05d54b3070 Add Hive metadata ETL process 2015-12-21 12:07:08 -08:00
Zhen Chen
d6f77a6f06 fix ldap org hierachy is empty bug and minor changes 2015-12-16 17:01:55 -08:00
Zhen Chen
af04ff6efc add git file commit history etl 2015-12-11 11:02:29 -08:00
SunZhaonan
b83c0a4322 Exclude tests that need configurations 2015-11-19 14:56:20 -08:00
SunZhaonan
d5c3d87d00 Initial commit 2015-11-19 14:39:21 -08:00