Mars Lan
35c0781f59
Use WH_APP_FOLDER property as the base directory for temp files for various ETL jobs ( #451 )
...
* Use WH_APP_FOLDER property as the base directory for temp files in Oracle ETL, instead of the full path defined in wh_etl_job_property, which is often /var/tmp/{something}.
* Move common code to FileUtil.py and move Voldamort's temp files too.
* Move Kafaka ETL temp files.
* Move Espresso ETL temp files.
* Move Multiproduct ETL temp files.
* Move CodeSearch ETL temp files.
* Move teradata ETL temp files.
2017-07-10 13:42:51 -07:00
Yi (Alan) Wang
27d12973e0
Update Nuage Load to purge non-exist dataset owners ( #453 )
2017-07-10 13:42:51 -07:00
Seyi Adebajo
3f569dc4c2
adds additional patterns request_header and mobile_header to tracking field regular expression. extracts tracking field pattern check to utils
2017-07-10 13:42:51 -07:00
Seyi Adebajo
566e9038cd
adds browse ui features: entity navigation, styles. routes for browse.entity. initial tests
2017-07-10 13:42:51 -07:00
Seyi Adebajo
3a28e294bb
updates datasets-list to integrate with the updated redux store keys
2017-07-10 13:42:51 -07:00
Mars Lan
f5a7e0c9ec
Make sure all intermediate directories are created for ETL job property files. ( #450 )
2017-07-10 13:42:51 -07:00
Seyi Adebajo
b610385542
abstracts common functionality between datasets/flows/metrics into entities modules for actions and reducers.
2017-07-10 13:42:50 -07:00
Yi (Alan) Wang
dbebb9fa4b
add owner_source to primary key in dataset_owner table schema ( #449 )
2017-07-10 13:42:50 -07:00
Seyi Adebajo
674b37aa38
adds actions for browse feature. abstracts reducers for datasets/metrics/flowsp into entities reducers. adds route for browse feature.
2017-07-10 13:42:50 -07:00
Seyi Adebajo
a3a135372e
refactors await use for proper case
2017-07-10 13:42:50 -07:00
Seyi Adebajo
b7e1a9d3aa
integrate ember-redux with wherehows. creates reducers and initial actions for datasets using FSA. creates container and presentational components for datasets-list. adds fetch polyfill for fetch api. updates yarn.lock. creates action-set to ensure action rules and conventions
2017-07-10 13:42:50 -07:00
Seyi Adebajo
33c4c472e5
adds ember-redux package as dep. removes exception handler in search route
2017-07-10 13:42:50 -07:00
Seyi Adebajo
c71d970846
sort bower json
2017-07-10 13:42:50 -07:00
Mars Lan
2499ee0116
Unify the open-source application.conf with internal ones so we don't need to maintain both.
2017-07-10 13:42:50 -07:00
Mars Lan
fda572dd8a
Allow the logback directory for ETL jobs to be overridden using system property ( #448 )
...
* Allow the logback directory for ETL jobs to be overridden using system property.
See https://logback.qos.ch/manual/configuration.html#variableSubstitution for more details.
* Add WHZ_ETL_TEMP_DIR env var and play config to control where the ETL job logs & temp files to be saved.
This enables us to move away from the default /var/tmp/wherehows directory.
2017-07-10 13:42:16 -07:00
Mars Lan
b2f0eaaf9d
Add app.version play config and have the value exposed via the config API. ( #447 )
2017-07-10 13:42:16 -07:00
Yi (Alan) Wang
9e95091ca7
return modified time in dataset ownership API ( #446 )
2017-07-10 13:42:16 -07:00
Seyi Adebajo
9d564d6a91
prefilled tracking fields are not rendered on the confidential classification tab
2017-07-10 13:42:16 -07:00
Seyi Adebajo
afd5e9f115
prefilled tracking fields are not rendered on the compliance classification tab
2017-07-10 13:42:16 -07:00
Mars Lan
32d8dd8ef9
Set the correct system property for krb5 config flie. ( #443 )
...
Clean up ember build properly
2017-07-10 13:42:16 -07:00
Na Zhang
973fb93167
Elastic search completion suggester for type ahead
2017-07-10 13:42:16 -07:00
Yi (Alan) Wang
674e6f728f
Add dataset level confidentiality field in security API and DB ( #441 )
2017-07-10 13:42:16 -07:00
Yi Wang
338fe52f3c
Add check for user not in session, at get current user API
2017-07-10 13:42:15 -07:00
Yi (Alan) Wang
6facc830cc
Check Linkedin interanl flag before fetching schema from Metastore ( #440 )
...
Also during column data mapping, check if native data type and description exists in the returned schema fields from Metadata store.
2017-07-10 13:42:15 -07:00
Yi (Alan) Wang
fe0d2ea5f9
Modify multiproduct ETL to set is_active for owner, return is_active to frontend ownership API ( #437 )
2017-07-10 13:42:15 -07:00
Seyi Adebajo
0199474138
updates placeholder text in dataset-table instances
2017-07-10 13:42:15 -07:00
Mars Lan
9b5cd71581
Change config tracking.piwik.siteid from type int to string. ( #436 )
...
This allows it to be set via env var (hence cfg2).
2017-07-10 13:42:15 -07:00
Seyi Adebajo
9939149adc
updates network request handlers to integrate with api changes for /compliance and /security endpoints
2017-07-10 13:42:15 -07:00
Yi (Alan) Wang
3d6c8188b1
Increase privacy and security table field size to save larger data ( #434 )
2017-07-10 13:42:15 -07:00
Seyi Adebajo
c20ab5a054
creates top-level application error substate
2017-07-10 13:42:15 -07:00
Seyi Adebajo
e7b151dd48
updates list of types for field format on confidential tab
2017-07-10 13:42:15 -07:00
Seyi Adebajo
8bfae8ca6f
fixes stream closed exception
2017-07-10 13:42:15 -07:00
Seyi Adebajo
55b62b3ae5
clear invalid test
2017-07-10 13:42:15 -07:00
Seyi Adebajo
8e41333b6c
changes wording on compliance tab. adds page count to dataset-table-pager component. removes index.scala.html
2017-07-10 13:42:15 -07:00
Seyi Adebajo
5ced597fc7
reinstates userId tracking, search tracking using the non default trackSiteSearch api
2017-07-10 13:42:14 -07:00
Seyi Adebajo
635551db6d
adds page tracking for user metrics. adds ember-metrics module. add integration for internal piwik tracking.
2017-07-10 13:42:14 -07:00
Seyi Adebajo
fd8a3acc7c
serves index.html from play packaged jar
2017-07-10 09:58:44 -07:00
Mars Lan
ebeda9f690
Remove unused env var from template.
2017-07-10 09:58:44 -07:00
Seyi Adebajo
448785c449
reinstates userId tracking, search tracking using the non default trackSiteSearch api
2017-07-10 09:58:44 -07:00
Seyi Adebajo
91820bd31a
adds page tracking for user metrics. adds ember-metrics module. add integration for internal piwik tracking.
2017-07-10 09:58:44 -07:00
Yi (Alan) Wang
e59005fb38
Add compliance and security API to frontend, remove dependency on backend ( #424 )
2017-07-10 09:58:44 -07:00
Seyi Adebajo
4063592230
refactors tracking props
2017-07-10 09:58:44 -07:00
Seyi Adebajo
65fcc7e4dd
changes package variables to local variable and moves responsibility to single function
2017-07-10 09:58:44 -07:00
Seyi Adebajo
863ba2bfda
adds endpoint for application config options
2017-07-10 09:58:43 -07:00
Yi (Alan) Wang
e729526486
Add metadata store access functions, update UrnUtil ( #422 )
2017-07-10 09:58:43 -07:00
Seyi Adebajo
ca12a859cc
updates Actions for other Play routes to serve index file. adds check for build output. updates routes to point to index Action and makes serveAsset private method
2017-07-10 09:58:43 -07:00
Mars Lan
9e55d80538
Add WHZ_KRB5_DIR environmental variable to the search path for gss-jass.conf & krb5.conf files. ( #421 )
...
Also remove the unset WH_HOME directory from the search path.
2017-07-10 09:58:43 -07:00
Seyi Adebajo
12f2bc9ec5
updates play to read output from build dir and serve content inline for serveAsset action. sets the rootURL for Ember to assets on prod. overrides ESA to redirect to / rather than default rootURL for an environment.
2017-07-10 09:58:43 -07:00
Seyi Adebajo
ae9162aa86
fixes regression with dataset-author name field
2017-07-10 09:58:43 -07:00
Yi (Alan) Wang
0643e4448a
Fix Hive Extract job file writing issues, add more authentication log ( #419 )
2017-07-10 09:58:43 -07:00
