Yi (Alan) Wang
34fe3b26a2
Merge pull request #216 from alyiwang/master
...
Add MatadataChangeEvent processor to call seperate APIs
2016-09-09 17:16:14 -07:00
jerrybai2009
2e428e9b28
Merge pull request #220 from jerrybai2009/master
...
fix the issue that can not delete and edit the comments
2016-09-09 16:56:06 -07:00
jbai
cd942e852c
fix the issue that can not delete and edit the comments
2016-09-09 10:42:46 -07:00
Yi Wang
a69d9b109a
fix a typo in dict_field_detail DDL
2016-09-08 09:34:29 -07:00
Yi Wang
5515cbdde9
Add MatadataChangeEvent processor to call seperate APIs
2016-09-06 16:41:50 -07:00
Yi (Alan) Wang
d04abe78c8
Merge pull request #212 from alyiwang/master
...
Map scm repo owner to dataset owner table
2016-09-06 11:19:24 -07:00
Yi Wang
4c500402fe
Map repo owner fix, change 'main' to 'Producer' and reset sort id
2016-09-02 13:52:00 -07:00
jerrybai2009
9bbbf1a68e
Merge pull request #213 from jerrybai2009/master
...
fix the issue that action getSchema did not fired
2016-09-02 11:13:22 -07:00
Eric Sun
0ac00e1af3
Update README.md
2016-09-02 09:35:13 -07:00
Douglas Moore
d44c194529
Backend service readme ( #215 )
...
* Update README.md
* Update README
* Rename README to README.me
* Rename README.me to README.md
* Update README.md
2016-09-02 09:31:52 -07:00
Yi Wang
a809b0ac47
Map repo owner fix to use dataset group mapping
2016-09-01 18:19:41 -07:00
jbai
f4f4538408
fix the issue that action getSchema did not fired
2016-08-31 16:50:20 -07:00
Yi Wang
81f891bfab
Map scm repo owner to dataset owner table
2016-08-30 15:35:28 -07:00
Yi (Alan) Wang
b8e9ff5a7c
Merge pull request #209 from alyiwang/master
...
Update DatasetOwnerRecord to be compatible with linkedin branch
2016-08-29 17:24:06 -07:00
Yi Wang
e2b42d2ccb
Update DatasetOwnerRecord to be compatible with linkedin branch
2016-08-25 09:12:31 -07:00
Yi Wang
183a9dcb6d
Merge branch 'master' of https://github.com/linkedin/WhereHows
2016-08-24 09:20:27 -07:00
Yi (Alan) Wang
579b8fc9d7
Add metadataChangeEvent APIs to backend-service ( #205 )
...
* Add multiproduct and git repo metadata etl job
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
* merge API tables to existing dataset owner and schema field table
* Add confidential and recursive column to dict_dataset_field
2016-08-24 09:10:35 -07:00
Yi Wang
7cbda15b5a
Add confidential and recursive column to dict_dataset_field
2016-08-23 15:50:30 -07:00
Yi Wang
d46a9d8b8e
merge API tables to existing dataset owner and schema field table
2016-08-22 17:06:20 -07:00
Yi Wang
46871face6
Add metadataChangeEvent APIs to backend-service
2016-08-16 18:47:53 -07:00
jerrybai2009
066c5f2ca5
Merge pull request #204 from jerrybai2009/master
...
add the ui test for javascript code
2016-08-16 15:47:33 -07:00
jbai
2389b6034a
add the ui test for javascript code
2016-08-15 23:35:37 -07:00
Yi Wang
38ae6a0276
Merge branch 'master' of https://github.com/linkedin/WhereHows
2016-08-12 12:27:20 -07:00
Yi (Alan) Wang
078e90e8bd
Add multiproduct and git repo metadata etl job ( #202 )
...
* Add multiproduct and git repo metadata etl job
* implement the dataset availability section
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
2016-08-12 12:26:55 -07:00
Yi Wang
44807f5f7e
Fix the incompatibility between integer repo_id in db and string field in record
2016-08-10 17:24:03 -07:00
Yi Wang
11158e0b9f
Remove unnecessary log entries from kafka processor
2016-08-10 11:23:39 -07:00
Yi Wang
bc276274ff
Use FileWriter to write records into CSV file
2016-08-10 11:20:31 -07:00
Yi Wang
83834e4e88
Add fetching acl owner info from svn, also change some property names.
2016-08-10 09:11:37 -07:00
jerrybai2009
162892a9e8
Merge pull request #203 from jerrybai2009/master
...
implement the dataset accessiblities section
2016-08-09 16:25:57 -07:00
Yi Wang
4689160dbb
Extract commit hash use it when querying acl
2016-08-09 13:05:59 -07:00
jbai
45c528c9d9
implement the dataset accessiblities section
2016-08-09 11:34:46 -07:00
Yi Wang
830413e122
Add multiproduct and git repo metadata etl job
2016-08-08 21:28:37 -07:00
Eric Sun
cd4853d0a5
Use ProcessBuilder and redirected log file for HDFS Extract ( #198 )
...
* Use ProcessBuilder and redirected log file for HDFS Extract
* relax urn validation rule
2016-08-08 14:02:34 -07:00
jerrybai2009
39cec22e25
Merge pull request #197 from jerrybai2009/master
...
upgrade ember from 1.12 to 2.6.2
2016-08-05 09:52:03 -07:00
jbai
5305124d8c
fix the bind-attr and wrong flow link in search result issues
2016-08-04 18:46:33 -07:00
jbai
23910971a9
upgrade ember from 1.12 to 2.6.2
2016-08-04 17:44:47 -07:00
Eric Sun
c4d1605a0c
Merge pull request #196 from alyiwang/master
...
Modify Kafka Master to handle more than one Kafka connection configurations
Add additional error handling when starting the service
With this change in place.
- each Kafka Zookeeper requires a corresponding entry defined in wh_etl_job
- the connection info (such as Zookeeper, SchemaRegistery, topic to staging table mapping...) are configured in wh_etl_job_property
- kafka.consumer.etl.jobid in application.conf will determine if such Kafka job will be launched when the backend-service starts
2016-08-04 14:26:59 -07:00
Yi Wang
c0cfe1f5ca
Modify KafkaConsumerMaster to handle more than one kafka config, add error handling
2016-08-04 13:07:19 -07:00
Eric Sun
ef584552be
Merge pull request #194 from alyiwang/master
...
Get cluster info from cfg_cluster and format kafka events cluster field
2016-08-03 20:24:07 -07:00
Yi Wang
3d3b2a8075
Get kafka job id from applicatoin.conf and then get ref_id and configs from DB
2016-08-03 18:55:07 -07:00
Yi Wang
dbbdb6e2fb
Modify Oracle metadata ETL job, use Json dumps and remove unnecessary quotes
2016-08-03 18:49:00 -07:00
jerrybai2009
b4a718efd0
Merge pull request #195 from ericsun2/master
...
temp fix for hdfs_schema_crawler getRuntime().exec() hangs problem
2016-08-03 18:15:43 -07:00
jerrybai2009
e7c7175cba
Merge pull request #188 from jerrybai2009/master
...
load the teradata and hadoop data into table dict_dataset_instance
2016-08-03 18:13:06 -07:00
Eric Sun
ca7542ca40
remove duplicate section
2016-08-03 17:52:49 -07:00
Eric Sun
53d40c8392
add a few new hdfs directory patterns
2016-08-03 16:16:58 -07:00
Eric Sun
1cd5872369
temp fix for hdfs_schema_crawler getRuntime().exec() hangs problem; exclude log4j
2016-08-03 15:50:00 -07:00
jerrybai2009
42431fea4b
Merge pull request #192 from ericsun2/master
...
add log_dataset_instance_load_status
2016-08-01 15:07:06 -07:00
Eric Sun
8c9cb99ba4
primary_dataset_type for cfg_database
2016-08-01 13:20:04 -07:00
Eric Sun
67479eac58
log_dataset_instance_load_status
2016-08-01 13:00:39 -07:00
Eric Sun
3553ad6a38
Merge pull request #189 from ericsun2/master
...
add python module [requests] for simple REST client
2016-07-29 23:17:28 -07:00
