Mars Lan
deb98480a3
Completely retire wh_property table and associated codes. ( #583 )
2017-07-10 13:44:35 -07:00
Mars Lan
0e5e5685d7
Do not automatically kick off all jobs that yet have an entry in wh_etl_job_schedule table. ( #582 )
...
* Do not automatically kick off all jobs that yet have an entry in wh_etl_job_schedule table.
Instead simply schedule an entry and wait for its time to come.
* Make the ETL actor pool size configurable and set the default to something more conservative.
2017-07-10 13:44:35 -07:00
Mars Lan
33f49c5f55
Fix more places where connection URLs are manually constructed. ( #579 )
2017-07-10 13:44:35 -07:00
Mars Lan
af37b3c39f
Rely on job file specifying kerberos.keytab.file using absolute path. ( #578 )
2017-07-10 13:44:35 -07:00
Mars Lan
932aca7f05
Call the correct method to create connection instead of buliding the connection string manually. ( #577 )
2017-07-10 13:44:35 -07:00
Mars Lan
770ac152e9
Refactor tree jobs to use job files. ( #576 )
2017-07-10 13:44:35 -07:00
Mars Lan
3f85b26d81
Drop the duplicated DB-related settings from wh_property table. ( #575 )
2017-07-10 13:44:35 -07:00
Mars Lan
9dca733d76
Move elastic-serach related props from wh_property table to job files. ( #574 )
...
This will break FlowTreeBuilder & DatasetTreeBuilder, both are not being used anyway.
2017-07-10 13:44:34 -07:00
Mars Lan
d2d92367f9
Also include TD table with O kind (no primary key) during ETL. ( #571 )
2017-07-10 13:44:34 -07:00
Mars Lan
e411aecf2b
Missed another TD ut_ table filter. ( #570 )
2017-07-10 13:44:34 -07:00
Mars Lan
9f39f1f380
Remove the filter for TD ut_* tables as we do need the schema for them. ( #569 )
2017-07-10 13:44:34 -07:00
Yi (Alan) Wang
4f9f7fc513
Add a few more fields to frontend metric batch API ( #567 )
2017-07-10 13:44:34 -07:00
Mars Lan
68bb73f4a7
Remove the custom access count filter for DM_BIZ etc. ( #568 )
2017-07-10 13:44:34 -07:00
Seyi Adebajo
8187293d65
adds support for constraining ui feature behind config flag - isInternal
2017-07-10 13:44:34 -07:00
Seyi Adebajo
f4328bab70
adds internal flag check for branding
2017-07-10 13:44:34 -07:00
Seyi Adebajo
d3b7e7bc2f
adds application configuration to application start sequence. create configurator service for application.
2017-07-10 13:44:34 -07:00
Seyi Adebajo
971b1001b0
updates the index page ui with feature icons
2017-07-10 13:44:34 -07:00
Seyi Adebajo
db1136f722
fixes empty feedback route: changed to mailto anchor
2017-07-10 13:44:34 -07:00
Seyi Adebajo
aaf304ec4f
updates newly added owner with confirmation flag as default value. updates default owner type to Owner
2017-07-10 13:44:33 -07:00
Seyi Adebajo
99dd51df48
port lineage for datasets from old-wh using dagre-d3
2017-07-10 13:44:33 -07:00
Mars Lan
75f57c6ac0
Remove some of the LI-speicifc ETLs. ( #559 )
2017-07-10 13:44:33 -07:00
Mars Lan
7c50a04f23
Add some example field formats that the user may have missed when saving compliance data. ( #558 )
2017-07-10 13:44:33 -07:00
Yi (Alan) Wang
5808179017
Increase play http parser buffer size to 10MB ( #555 )
2017-07-10 13:44:33 -07:00
Seyi Adebajo
08e12d886f
fixes issue with file upload of compliance json on save attempt
2017-07-10 13:44:33 -07:00
Yi (Alan) Wang
98ffbfb65f
track last modified person of compliance info ( #554 )
2017-07-10 13:44:33 -07:00
Mars Lan
ea1a11d01c
Add settings dataset-level tag. ( #549 )
2017-07-10 13:44:33 -07:00
Mars Lan
90f59d23eb
Replace tooltip for ID types with link to Wiki. ( #548 )
2017-07-10 13:44:33 -07:00
Seyi Adebajo
c91ca85a90
updates the behaviour for custom identifier types to set a default classification
2017-07-10 13:44:33 -07:00
Mars Lan
53a30d5a77
Major refactoring of ETL scheduling & configuration ( #542 )
...
* Major refactoring
- Move job-spcific properties from wh_etl_job_property table to .job files
- Use the job file name instead of numeric IDs to identify ETL jobs
- Use reflection to create ETL job class at run time instead of relying hard-coded enums
- Drop ETL job-related APIs as they're no longer needed
- Drop wh_etl_job, wh_etl_job_property, wh_etl_job_execution tables
- Add wh_etl_schedule & wh_etl_history tables
2017-07-10 13:44:33 -07:00
Seyi Adebajo
9ea4545db5
updates file upload to autosave policy if checks pass
2017-07-10 13:43:00 -07:00
Seyi Adebajo
e2cf19a46a
adds support for custom id in dataset compliance. adds divider for none type in options list. adds edit mode.
2017-07-10 13:43:00 -07:00
Seyi Adebajo
c3a53b4f0b
adds support to disable options on select. removes lowercase transformation on string option values: consumer responsibility
2017-07-10 13:43:00 -07:00
Yi (Alan) Wang
50c1901890
Add dataset name convert for dalids and hive ( #538 )
...
* Add dataset name convert for dalids and hive
* Update javadoc for dataset name convert
2017-07-10 13:43:00 -07:00
Seyi Adebajo
b7895fe5bb
fixes wrong reference to new policy flag. fixes missing model name in downloaded json file.
2017-07-10 13:43:00 -07:00
Seyi Adebajo
c3de4a5ff7
adds feature to download a compliance policy as a json document. upload as json and apply to policy. performs attribute type assertions on values in uploaded policy
2017-07-10 13:43:00 -07:00
Mars Lan
30f9bf28b2
Filter out HDFS datasets with "temporary" in path name. ( #537 )
...
Treat "_" as a word boundary.
2017-07-10 13:43:00 -07:00
Mars Lan
8411e220a4
Actually write properties to dict_dataset table. ( #529 )
...
Temporariliy disable the broken updateKafkaDatasetOwner call.
Add extra blacklist pattern.
2017-07-10 13:43:00 -07:00
Yi (Alan) Wang
445a574a87
Change compliance column type to MEDIUMTEXT ( #530 )
2017-07-10 13:43:00 -07:00
Seyi Adebajo
d7a5b425ff
updates compliance wording on tooltips and updates links
2017-07-10 13:43:00 -07:00
Seyi Adebajo
287796b933
refactors nonIdFieldLogicalTypes decoupling stored representation from presentational values
2017-07-10 13:43:00 -07:00
Mars Lan
b486f7ee38
Extract and set dataset name correctly from Gobblin metadata audit event. ( #526 )
2017-07-10 13:42:59 -07:00
Seyi Adebajo
8ab2ce9252
fixes issue with tracking current user immediately after login: moves user tracking to index from application. adds session invalidation when retrieving the current user fails. slightly refactors logout
2017-07-10 13:42:59 -07:00
Seyi Adebajo
23c1ff5d78
removes obsolete confidential feature component sources
2017-07-10 13:42:59 -07:00
Mars Lan
d3a92b7d2c
Expand GobblinTrackingAuditProcessor to handle metadata audit events. ( #521 )
...
* Expand GobblinTrackingAuditProcessor to handle metadata audit events.
* Fix formating
2017-07-10 13:42:59 -07:00
Seyi Adebajo
3233fb91e4
adds default value for mixed identifier types. disabled field format select for mixed identifier types. removes geolocation from field format options. updates defaulting logic for mixed types
2017-07-10 13:42:59 -07:00
Seyi Adebajo
22625a119b
adds prettier precommit hook for app js files
2017-07-10 13:42:59 -07:00
Yi (Alan) Wang
fcbde02b37
Stop fetching EI/DEV-only Espresso DB ( #522 )
2017-07-10 13:42:59 -07:00
Seyi Adebajo
86570c7ca8
fixes debouncing issue: invalid function referencing
2017-07-10 13:42:59 -07:00
Seyi Adebajo
4a66c1313a
lines left over from last commit: adds urn as singular option for mixed type
2017-07-10 13:42:59 -07:00
Seyi Adebajo
56417b2347
adds browser confirmation dialog for unformatted non-id fields pending mocks and time. sorts options for non-id field formats. splits ge location option. restricts field format to urn for mixed identifier type.
2017-07-10 13:42:59 -07:00
