Mars Lan
0d712c07d1
Update README.md
2017-07-10 17:14:34 -07:00
Mars Lan
cd80307463
Fix broken build due to bad merge.
2017-07-10 17:05:14 -07:00
Mars Lan
70f753d895
Update README.md
2017-07-10 16:56:36 -07:00
Mars Lan
120bd7c4bc
Rename wherehows-api to wherehows-frontend to be consistent with the actual artifact. ( #589 )
2017-07-10 13:44:36 -07:00
Mars Lan
16a95e6ed5
Clean up LI-specific constants and code. ( #588 )
2017-07-10 13:44:35 -07:00
Mars Lan
c7b6fd1688
Move TMS restli related code out of open source. ( #587 )
...
Add skeleton for a generalized DAO framework.
2017-07-10 13:44:35 -07:00
Mars Lan
86b815eadf
Fix broken Travis build.
2017-07-10 13:44:35 -07:00
Mars Lan
bb5f483be9
Clean up comments and DDL for obsolete tables. ( #586 )
2017-07-10 13:44:35 -07:00
Mars Lan
a950cdbc1a
Fix the bug where MySQL credential isn't properly passed to DatabaseWriter. ( #585 )
2017-07-10 13:44:35 -07:00
Mars Lan
8058734917
Fix the last few instances whewre username & password are specified via manual construction of url. ( #584 )
2017-07-10 13:44:35 -07:00
Mars Lan
deb98480a3
Completely retire wh_property table and associated codes. ( #583 )
2017-07-10 13:44:35 -07:00
Mars Lan
0e5e5685d7
Do not automatically kick off all jobs that yet have an entry in wh_etl_job_schedule table. ( #582 )
...
* Do not automatically kick off all jobs that yet have an entry in wh_etl_job_schedule table.
Instead simply schedule an entry and wait for its time to come.
* Make the ETL actor pool size configurable and set the default to something more conservative.
2017-07-10 13:44:35 -07:00
Mars Lan
33f49c5f55
Fix more places where connection URLs are manually constructed. ( #579 )
2017-07-10 13:44:35 -07:00
Mars Lan
af37b3c39f
Rely on job file specifying kerberos.keytab.file using absolute path. ( #578 )
2017-07-10 13:44:35 -07:00
Mars Lan
932aca7f05
Call the correct method to create connection instead of buliding the connection string manually. ( #577 )
2017-07-10 13:44:35 -07:00
Mars Lan
770ac152e9
Refactor tree jobs to use job files. ( #576 )
2017-07-10 13:44:35 -07:00
Mars Lan
3f85b26d81
Drop the duplicated DB-related settings from wh_property table. ( #575 )
2017-07-10 13:44:35 -07:00
Mars Lan
9dca733d76
Move elastic-serach related props from wh_property table to job files. ( #574 )
...
This will break FlowTreeBuilder & DatasetTreeBuilder, both are not being used anyway.
2017-07-10 13:44:34 -07:00
Mars Lan
d2d92367f9
Also include TD table with O kind (no primary key) during ETL. ( #571 )
2017-07-10 13:44:34 -07:00
Mars Lan
e411aecf2b
Missed another TD ut_ table filter. ( #570 )
2017-07-10 13:44:34 -07:00
Mars Lan
9f39f1f380
Remove the filter for TD ut_* tables as we do need the schema for them. ( #569 )
2017-07-10 13:44:34 -07:00
Yi (Alan) Wang
4f9f7fc513
Add a few more fields to frontend metric batch API ( #567 )
2017-07-10 13:44:34 -07:00
Mars Lan
68bb73f4a7
Remove the custom access count filter for DM_BIZ etc. ( #568 )
2017-07-10 13:44:34 -07:00
Seyi Adebajo
8187293d65
adds support for constraining ui feature behind config flag - isInternal
2017-07-10 13:44:34 -07:00
Seyi Adebajo
f4328bab70
adds internal flag check for branding
2017-07-10 13:44:34 -07:00
Seyi Adebajo
d3b7e7bc2f
adds application configuration to application start sequence. create configurator service for application.
2017-07-10 13:44:34 -07:00
Seyi Adebajo
971b1001b0
updates the index page ui with feature icons
2017-07-10 13:44:34 -07:00
Seyi Adebajo
db1136f722
fixes empty feedback route: changed to mailto anchor
2017-07-10 13:44:34 -07:00
Seyi Adebajo
aaf304ec4f
updates newly added owner with confirmation flag as default value. updates default owner type to Owner
2017-07-10 13:44:33 -07:00
Seyi Adebajo
99dd51df48
port lineage for datasets from old-wh using dagre-d3
2017-07-10 13:44:33 -07:00
Mars Lan
75f57c6ac0
Remove some of the LI-speicifc ETLs. ( #559 )
2017-07-10 13:44:33 -07:00
Mars Lan
7c50a04f23
Add some example field formats that the user may have missed when saving compliance data. ( #558 )
2017-07-10 13:44:33 -07:00
Yi (Alan) Wang
5808179017
Increase play http parser buffer size to 10MB ( #555 )
2017-07-10 13:44:33 -07:00
Seyi Adebajo
08e12d886f
fixes issue with file upload of compliance json on save attempt
2017-07-10 13:44:33 -07:00
Yi (Alan) Wang
98ffbfb65f
track last modified person of compliance info ( #554 )
2017-07-10 13:44:33 -07:00
Mars Lan
ea1a11d01c
Add settings dataset-level tag. ( #549 )
2017-07-10 13:44:33 -07:00
Mars Lan
90f59d23eb
Replace tooltip for ID types with link to Wiki. ( #548 )
2017-07-10 13:44:33 -07:00
Seyi Adebajo
c91ca85a90
updates the behaviour for custom identifier types to set a default classification
2017-07-10 13:44:33 -07:00
Mars Lan
53a30d5a77
Major refactoring of ETL scheduling & configuration ( #542 )
...
* Major refactoring
- Move job-spcific properties from wh_etl_job_property table to .job files
- Use the job file name instead of numeric IDs to identify ETL jobs
- Use reflection to create ETL job class at run time instead of relying hard-coded enums
- Drop ETL job-related APIs as they're no longer needed
- Drop wh_etl_job, wh_etl_job_property, wh_etl_job_execution tables
- Add wh_etl_schedule & wh_etl_history tables
2017-07-10 13:44:33 -07:00
Seyi Adebajo
9ea4545db5
updates file upload to autosave policy if checks pass
2017-07-10 13:43:00 -07:00
Seyi Adebajo
e2cf19a46a
adds support for custom id in dataset compliance. adds divider for none type in options list. adds edit mode.
2017-07-10 13:43:00 -07:00
Seyi Adebajo
c3a53b4f0b
adds support to disable options on select. removes lowercase transformation on string option values: consumer responsibility
2017-07-10 13:43:00 -07:00
Yi (Alan) Wang
50c1901890
Add dataset name convert for dalids and hive ( #538 )
...
* Add dataset name convert for dalids and hive
* Update javadoc for dataset name convert
2017-07-10 13:43:00 -07:00
Seyi Adebajo
b7895fe5bb
fixes wrong reference to new policy flag. fixes missing model name in downloaded json file.
2017-07-10 13:43:00 -07:00
Seyi Adebajo
c3de4a5ff7
adds feature to download a compliance policy as a json document. upload as json and apply to policy. performs attribute type assertions on values in uploaded policy
2017-07-10 13:43:00 -07:00
Mars Lan
30f9bf28b2
Filter out HDFS datasets with "temporary" in path name. ( #537 )
...
Treat "_" as a word boundary.
2017-07-10 13:43:00 -07:00
Mars Lan
8411e220a4
Actually write properties to dict_dataset table. ( #529 )
...
Temporariliy disable the broken updateKafkaDatasetOwner call.
Add extra blacklist pattern.
2017-07-10 13:43:00 -07:00
Yi (Alan) Wang
445a574a87
Change compliance column type to MEDIUMTEXT ( #530 )
2017-07-10 13:43:00 -07:00
Seyi Adebajo
d7a5b425ff
updates compliance wording on tooltips and updates links
2017-07-10 13:43:00 -07:00
Seyi Adebajo
287796b933
refactors nonIdFieldLogicalTypes decoupling stored representation from presentational values
2017-07-10 13:43:00 -07:00
