15 Commits

Author	SHA1	Message	Date
jbai	0c68d9c4fb	fix the dataset field has duplicated records issue	2016-06-16 16:33:37 -07:00
jbai	38fdf1c132	fix the dalids dependency issue and add more log info in elasticsearch	2016-06-16 14:38:01 -07:00
jbai	5062fa4ecc	load dali depends on and instance into final table	2016-06-09 18:51:05 -07:00
jbai	f344f1cf2e	update code to follow the code review	2016-06-07 11:39:07 -07:00
jbai	e5880cf81a	fix the merge conflict	2016-06-07 11:31:00 -07:00
SunZhaonan	310e6e9f06	Catch hive table even though its schema is None or exception message	2016-06-03 11:30:08 -07:00
jbai	af976c3d5e	Dali Metadata integration - combine dali versions into one node	2016-06-02 18:29:44 -07:00
SunZhaonan	fb1198e4ae	Fix hive field ETL bug	2016-05-20 17:52:45 -07:00
SunZhaonan	a0b7cb9d57	Fix process hanging bug. Add hive field ETL process.	2016-03-16 19:12:21 -07:00
SunZhaonan	aff8f323e4	Scheduler check previous job is finished. Redirect remote outputstream into log. Fix avro parser bugs	2016-03-16 19:09:53 -07:00
SunZhaonan	6b024196cd	Fix Hive extract disorder bug. Add Hive database optional whitelist params	2016-03-16 19:09:52 -07:00
SunZhaonan	5e9ae37952	Change to multi processing instead of multi thread. Fix hive ETL bug	2016-02-29 16:37:03 -08:00
SunZhaonan	c3da00003e	Fix bugs. Reenforce logging. Format jython scripts. Add missing table DDL.	2016-02-03 19:22:18 -08:00
SunZhaonan	0eddd8accf	close connection appropriatly, minior fix	2015-12-21 12:07:08 -08:00
SunZhaonan	05d54b3070	Add Hive metadata ETL process	2015-12-21 12:07:08 -08:00