Mars Lan
082e882268
Remove now-obsolete Kafka-related code from wherehows-backend ( #738 )
2017-09-11 17:49:19 -07:00
Yi (Alan) Wang
cd5f72b7ee
Rename DAO and model into table and view sub-package ( #722 )
2017-09-05 09:59:35 -07:00
hzhang2
2ca851753a
remove backend changes for this PR
2017-08-22 22:35:27 -07:00
Yi (Alan) Wang
28b83b8e7b
Add BaseJob in wherehows-common, make ETLjob extends from it ( #681 )
2017-08-16 21:24:38 -07:00
hzhang2
4eabd773d2
skip empty classify result DB insertion ( #668 )
2017-08-14 15:09:38 -07:00
Yi (Alan) Wang
3479180ca3
Fix DAO Factory config name error, refactor some code ( #658 )
2017-08-10 18:13:59 -07:00
hzhang2
c8dcd2ef83
submit hibernate cfg ( #636 )
2017-08-03 15:04:56 -07:00
Yi (Alan) Wang
d52edaf3e1
Allow ETL jobs to be disabled at run time ( #622 )
2017-07-26 21:13:56 -07:00
Yi (Alan) Wang
6a8fe66ef9
Add active, deprecated flag to dataset, modify ETL to update is_active ( #604 )
2017-07-19 17:07:28 -07:00
Mars Lan
bb5f483be9
Clean up comments and DDL for obsolete tables. ( #586 )
2017-07-10 13:44:35 -07:00
Mars Lan
deb98480a3
Completely retire wh_property table and associated codes. ( #583 )
2017-07-10 13:44:35 -07:00
Mars Lan
53a30d5a77
Major refactoring of ETL scheduling & configuration ( #542 )
...
* Major refactoring
- Move job-spcific properties from wh_etl_job_property table to .job files
- Use the job file name instead of numeric IDs to identify ETL jobs
- Use reflection to create ETL job class at run time instead of relying hard-coded enums
- Drop ETL job-related APIs as they're no longer needed
- Drop wh_etl_job, wh_etl_job_property, wh_etl_job_execution tables
- Add wh_etl_schedule & wh_etl_history tables
2017-07-10 13:44:33 -07:00
Mars Lan
30f9bf28b2
Filter out HDFS datasets with "temporary" in path name. ( #537 )
...
Treat "_" as a word boundary.
2017-07-10 13:43:00 -07:00
Mars Lan
8411e220a4
Actually write properties to dict_dataset table. ( #529 )
...
Temporariliy disable the broken updateKafkaDatasetOwner call.
Add extra blacklist pattern.
2017-07-10 13:43:00 -07:00
Mars Lan
b486f7ee38
Extract and set dataset name correctly from Gobblin metadata audit event. ( #526 )
2017-07-10 13:42:59 -07:00
Mars Lan
d3a92b7d2c
Expand GobblinTrackingAuditProcessor to handle metadata audit events. ( #521 )
...
* Expand GobblinTrackingAuditProcessor to handle metadata audit events.
* Fix formating
2017-07-10 13:42:59 -07:00
Mars Lan
5f5c0937d1
Rename web, backend-service ( #490 )
...
* Rename web to wherehows-api and update README.
* Rename backend-service to wherehows-backend
* Rename metadata-etl to wherehows-etl
* Rename hadoop-dataset-extractor-standalone to wherehows-hadoop
2017-07-10 13:42:56 -07:00
