Yi (Alan) Wang
dbebb9fa4b
add owner_source to primary key in dataset_owner table schema ( #449 )
2017-07-10 13:42:50 -07:00
Na Zhang
973fb93167
Elastic search completion suggester for type ahead
2017-07-10 13:42:16 -07:00
Yi (Alan) Wang
674e6f728f
Add dataset level confidentiality field in security API and DB ( #441 )
2017-07-10 13:42:16 -07:00
Yi (Alan) Wang
fe0d2ea5f9
Modify multiproduct ETL to set is_active for owner, return is_active to frontend ownership API ( #437 )
2017-07-10 13:42:15 -07:00
Yi (Alan) Wang
3d6c8188b1
Increase privacy and security table field size to save larger data ( #434 )
2017-07-10 13:42:15 -07:00
jbai
75b4d10526
add the table job attempt source code
2017-07-10 09:53:26 -07:00
Matthew Ropp
b0626fe957
myql57_pk_not_null - primary key columns must be not null ( #378 )
2017-03-30 08:44:18 -07:00
Yi (Alan) Wang
91926f9540
Fix table creation DDL bug for MySQL 5.7 ( #375 )
...
Primary key fields needs to be "NOT NULL".
2017-03-29 15:57:34 -07:00
Yi (Alan) Wang
d9438dc5bc
Fix SQL query issues when upgrading to MySQL 5.7 ( #345 )
2017-03-15 09:22:46 -07:00
Yi (Alan) Wang
66a8eea21b
Fix issues from Oracle MetadataChangeEvent integration ( #336 )
...
* Fix issues from Oracle MetadataChangeEvent integration
2017-03-14 17:19:30 -07:00
Yi (Alan) Wang
89c3b93d6f
Fix MySQL DDL error for stg_dict_dataset_instance ( #314 )
2017-01-27 11:35:41 -08:00
camelliazhang
724f754f03
clean and refactor elastic serach ETL job ( #300 )
2016-12-14 21:22:30 -08:00
Yi Wang
51f911b400
Map git repo and owners to Oracle/espresso/dali datasets
2016-11-22 10:51:00 -08:00
Yi Wang
6310d6c0e0
Update etl_config_tables DDL, add default timestamps
2016-11-14 15:23:55 -08:00
Yi (Alan) Wang
e07306b51e
Update MetadataChangeEvent, separate privacy compliance from security ( #275 )
2016-11-11 17:25:41 -08:00
Yi Wang
7a76bc5f59
Update MetadataChangeEvent, separate privacy compliance from security
2016-11-11 17:10:17 -08:00
Yi Wang
b4f5e438e2
Add JobExecutionLineageEvent and kafka processor
2016-11-08 19:11:37 -08:00
Stephane Maarek
a32a0e4b7f
last_modified column has a default for the table cfg_database ( #269 )
...
@alyiwang will also touch a few other cfg_??? tables for the default values
2016-11-08 17:45:27 -08:00
Na Zhang
217b7d9d09
search ranking improvement with static boosting
2016-11-08 15:18:51 -08:00
Na Zhang
043dc25e89
Get owners for espresso and oracle, and fix a bug for teradata
2016-10-19 11:13:32 -07:00
Yi Wang
3227412339
Login authentication support multiple LDAP servers, add login history
2016-10-13 14:30:43 -07:00
Yi Wang
c9f4f18d9c
Update Azkaban_Execution job to fetch cronExpression in flow scheduling
2016-10-06 13:43:10 -07:00
Yi (Alan) Wang
c9dfb637af
Update MetadataChangeEvent APIs according to schema change ( #243 )
...
* Update MetadataChangeEvent APIs according to schema change
* Update MultiproductLoad to reflect new Owner types
* Add comments for Owner_type precedence (priority) and compliance
2016-10-06 13:33:45 -07:00
Yi Wang
0356497124
Add comments for Owner_type precedence (priority) and compliance
2016-10-06 13:24:29 -07:00
Yi Wang
8ab5c824b0
Update MultiproductLoad to reflect new Owner types
2016-10-03 18:39:21 -07:00
Yi Wang
b74d58a33f
Update MetadataChangeEvent APIs according to schema change
2016-10-03 10:56:23 -07:00
Yi Wang
be65efb0cc
Update backend APIs to cast SQL results back to Java record then serialize to Json reply
2016-09-20 18:56:49 -07:00
jbai
44df5dd08a
update the dataset ownership information
2016-09-19 16:10:23 -07:00
Daniel Grove
937637c3ea
Add Database to create_tables readme ( #225 )
...
* Add Database to create_tables readme
* Add -D to signify database
2016-09-16 09:18:49 -07:00
Yi Wang
b136fc6c37
Add MetadataInventoryEvent processor and API
2016-09-15 09:22:42 -07:00
Yi Wang
5ce5a1425e
Add hostname and process_id to wh_etl_job_execution
2016-09-12 16:09:33 -07:00
Yi Wang
a69d9b109a
fix a typo in dict_field_detail DDL
2016-09-08 09:34:29 -07:00
Yi Wang
5515cbdde9
Add MatadataChangeEvent processor to call seperate APIs
2016-09-06 16:41:50 -07:00
Yi Wang
81f891bfab
Map scm repo owner to dataset owner table
2016-08-30 15:35:28 -07:00
Yi (Alan) Wang
579b8fc9d7
Add metadataChangeEvent APIs to backend-service ( #205 )
...
* Add multiproduct and git repo metadata etl job
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
* merge API tables to existing dataset owner and schema field table
* Add confidential and recursive column to dict_dataset_field
2016-08-24 09:10:35 -07:00
Yi (Alan) Wang
078e90e8bd
Add multiproduct and git repo metadata etl job ( #202 )
...
* Add multiproduct and git repo metadata etl job
* implement the dataset availability section
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
2016-08-12 12:26:55 -07:00
Eric Sun
67479eac58
log_dataset_instance_load_status
2016-08-01 13:00:39 -07:00
Eric Sun
50c222d478
Avro schema for Kafka topics
2016-07-28 16:55:47 -07:00
Eric Sun
f745642212
add datacenter, deploymenttier, cluster to describe dataset instance
2016-07-28 16:38:03 -07:00
Yi Wang
6d4706bc62
Ingest Gobblin tracking events into wherehows using Kafka consumer client
2016-07-25 15:03:29 -07:00
jbai
f3c299480f
update the column names from schema to schema_text and view_expanded_text to ddl_text
2016-07-20 18:01:25 -07:00
jbai
33b05cde4b
tracking the dalids schema and expanded text by versions
2016-07-20 15:59:11 -07:00
jbai
c1b96db103
make the column version_sort_id of table dict_dataset_instance it not null and default value is 0
2016-06-29 23:00:18 -07:00
jbai
9166db7563
update the dict_dataset_instance data loading sql since table key changed
2016-06-29 18:00:10 -07:00
Eric Sun
807e59330e
travis mysql setup
2016-06-22 21:21:19 -07:00
Eric Sun
56cebd0d9c
add dict_dataset_instance and cfg_object_name_map to track dataset on multiple clusters and their replication dependency and view dependency
2016-06-22 19:40:05 -07:00
jerrybai2009
8a9eeb1bb8
Merge pull request #126 from jerrybai2009/master
...
support the elasticsearch as search engine
2016-05-23 18:12:55 -07:00
jbai
a2e42d60f3
add the elasticsearch index build and update file
2016-05-23 17:58:37 -07:00
Rafal Kluszczynski
8501b1d7b8
fix: update DDL for job_execution_ext_reference tables to work with mysql 5.7.x
2016-05-23 10:37:30 +02:00
Eric Sun
5ff4d35f51
add cfg_object_name_map and job_execution_ext_reference
2016-05-19 12:39:06 -07:00
