12 Commits

Author SHA1 Message Date
Na Zhang
04664a8fec fix a type in application conf to allow run java 2017-11-09 15:42:05 -08:00
Mars Lan
bf16a411a8
Make the ETL java command configurable (#845) 2017-11-07 15:54:05 -08:00
Shridhar Sattur
4c3aaf5305 Updating ETL config to incorporate unify logging. (#805)
* Updating ETL config to incorporate unify logging.

* Update EtlJobActor.java

* Update EtlJobActor.java
2017-10-20 10:22:30 -07:00
hzhang2
2215c32e19 correct syntax in application.conf and correct default value for mysql5dialect (#644) 2017-08-08 10:56:28 -07:00
hzhang2
7e3d4c220b fix hibernate connection pool only support mysql issue (#641) 2017-08-07 17:26:37 -07:00
Shridhar Sattur
aa28eba78a Added an end point in wherehows-frontend to display all the libs from container directory (#639) 2017-08-07 13:55:05 -07:00
Mars Lan
ec0b5ac309 Fix and significatly simplify the docker build process. (#593) 2017-07-13 09:39:59 -07:00
Mars Lan
0e5e5685d7 Do not automatically kick off all jobs that yet have an entry in wh_etl_job_schedule table. (#582)
* Do not automatically kick off all jobs that yet have an entry in wh_etl_job_schedule table.
Instead simply schedule an entry and wait for its time to come.

* Make the ETL actor pool size configurable and set the default to something more conservative.
2017-07-10 13:44:35 -07:00
Yi (Alan) Wang
5808179017 Increase play http parser buffer size to 10MB (#555) 2017-07-10 13:44:33 -07:00
Mars Lan
53a30d5a77 Major refactoring of ETL scheduling & configuration (#542)
* Major refactoring
- Move job-spcific properties from wh_etl_job_property table to .job files
- Use the job file name instead of numeric IDs to identify ETL jobs
- Use reflection to create ETL job class at run time instead of relying hard-coded enums
- Drop ETL job-related APIs as they're no longer needed
- Drop wh_etl_job, wh_etl_job_property, wh_etl_job_execution tables
- Add wh_etl_schedule & wh_etl_history tables
2017-07-10 13:44:33 -07:00
Mars Lan
d3a92b7d2c Expand GobblinTrackingAuditProcessor to handle metadata audit events. (#521)
* Expand GobblinTrackingAuditProcessor to handle metadata audit events.

* Fix formating
2017-07-10 13:42:59 -07:00
Mars Lan
5f5c0937d1 Rename web, backend-service (#490)
* Rename web to wherehows-api and update README.

* Rename backend-service to wherehows-backend

* Rename metadata-etl to wherehows-etl

* Rename hadoop-dataset-extractor-standalone to wherehows-hadoop
2017-07-10 13:42:56 -07:00