Yi (Alan) Wang
579b8fc9d7
Add metadataChangeEvent APIs to backend-service ( #205 )
...
* Add multiproduct and git repo metadata etl job
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
* merge API tables to existing dataset owner and schema field table
* Add confidential and recursive column to dict_dataset_field
2016-08-24 09:10:35 -07:00
Eric Sun
cd4853d0a5
Use ProcessBuilder and redirected log file for HDFS Extract ( #198 )
...
* Use ProcessBuilder and redirected log file for HDFS Extract
* relax urn validation rule
2016-08-08 14:02:34 -07:00
Yi Wang
3d3b2a8075
Get kafka job id from applicatoin.conf and then get ref_id and configs from DB
2016-08-03 18:55:07 -07:00
Eric Sun
8c9cb99ba4
primary_dataset_type for cfg_database
2016-08-01 13:20:04 -07:00
Eric Sun
f745642212
add datacenter, deploymenttier, cluster to describe dataset instance
2016-07-28 16:38:03 -07:00
jbai
7a77aba4b7
merge the pull request 165 to master branch
2016-07-21 10:38:36 -07:00
Naga Srinivas Vemuri
97370ed2e1
Query Dataset properties to retrieve datasetUrns
2016-07-21 11:54:47 +05:30
jbai
6af54658d6
merge Fetching dataset watchers via get /dataset/watchers to main branch
2016-06-30 10:20:54 -07:00
jbai
6974ae26ae
fix the gradlew check failed issue and make the cluster name input is mandatory
2016-06-15 11:12:20 -07:00
jbai
9705a07ad8
provide the dataset dependency api
2016-06-14 16:17:24 -07:00
SunZhaonan
2431d29432
Fix dataset insert API bug
2016-03-18 15:27:43 -07:00
SunZhaonan
c4671d2579
Add field comments ETL
...
Fix API bug of tech_matrix_id
Add key in comment table
2016-03-14 14:23:33 -07:00
SunZhaonan
5e9ae37952
Change to multi processing instead of multi thread. Fix hive ETL bug
2016-02-29 16:37:03 -08:00
SunZhaonan
033a28faee
Backend job property API. 'update' change to 'insert on duplicate update'
2016-02-17 16:42:54 -08:00
SunZhaonan
05d54b3070
Add Hive metadata ETL process
2015-12-21 12:07:08 -08:00
SunZhaonan
cd44daba5d
Merge with master
2015-12-16 15:54:50 -08:00
SunZhaonan
07c46304b5
Fix bug of duplicate field loading. Fix bug of subflow process in azkaban lineage ETL.
2015-12-11 19:46:35 -08:00
Zhen Chen
e21a6b8e75
remove unused variable, change load script and add etl type in backend service
2015-12-11 13:52:23 -08:00
Zhen Chen
ebbf9ec629
add ldap user and group metadata etl
2015-12-10 16:26:57 -08:00
Zhen Chen
5a08134b8d
add dataset owner metadata etl
2015-12-07 15:17:01 -08:00
SunZhaonan
d5c3d87d00
Initial commit
2015-11-19 14:39:21 -08:00
