Yi (Alan) Wang
e07306b51e
Update MetadataChangeEvent, separate privacy compliance from security ( #275 )
2016-11-11 17:25:41 -08:00
Yi Wang
b4f5e438e2
Add JobExecutionLineageEvent and kafka processor
2016-11-08 19:11:37 -08:00
Stephane Maarek
a32a0e4b7f
last_modified column has a default for the table cfg_database ( #269 )
...
@alyiwang will also touch a few other cfg_??? tables for the default values
2016-11-08 17:45:27 -08:00
Na Zhang
217b7d9d09
search ranking improvement with static boosting
2016-11-08 15:18:51 -08:00
Na Zhang
043dc25e89
Get owners for espresso and oracle, and fix a bug for teradata
2016-10-19 11:13:32 -07:00
Yi Wang
3227412339
Login authentication support multiple LDAP servers, add login history
2016-10-13 14:30:43 -07:00
Yi Wang
c9f4f18d9c
Update Azkaban_Execution job to fetch cronExpression in flow scheduling
2016-10-06 13:43:10 -07:00
Yi (Alan) Wang
c9dfb637af
Update MetadataChangeEvent APIs according to schema change ( #243 )
...
* Update MetadataChangeEvent APIs according to schema change
* Update MultiproductLoad to reflect new Owner types
* Add comments for Owner_type precedence (priority) and compliance
2016-10-06 13:33:45 -07:00
Yi Wang
0356497124
Add comments for Owner_type precedence (priority) and compliance
2016-10-06 13:24:29 -07:00
Yi Wang
8ab5c824b0
Update MultiproductLoad to reflect new Owner types
2016-10-03 18:39:21 -07:00
Yi Wang
b74d58a33f
Update MetadataChangeEvent APIs according to schema change
2016-10-03 10:56:23 -07:00
Yi Wang
be65efb0cc
Update backend APIs to cast SQL results back to Java record then serialize to Json reply
2016-09-20 18:56:49 -07:00
jbai
44df5dd08a
update the dataset ownership information
2016-09-19 16:10:23 -07:00
Daniel Grove
937637c3ea
Add Database to create_tables readme ( #225 )
...
* Add Database to create_tables readme
* Add -D to signify database
2016-09-16 09:18:49 -07:00
Yi Wang
b136fc6c37
Add MetadataInventoryEvent processor and API
2016-09-15 09:22:42 -07:00
Yi Wang
5ce5a1425e
Add hostname and process_id to wh_etl_job_execution
2016-09-12 16:09:33 -07:00
Yi Wang
a69d9b109a
fix a typo in dict_field_detail DDL
2016-09-08 09:34:29 -07:00
Yi Wang
5515cbdde9
Add MatadataChangeEvent processor to call seperate APIs
2016-09-06 16:41:50 -07:00
Yi Wang
81f891bfab
Map scm repo owner to dataset owner table
2016-08-30 15:35:28 -07:00
Yi (Alan) Wang
579b8fc9d7
Add metadataChangeEvent APIs to backend-service ( #205 )
...
* Add multiproduct and git repo metadata etl job
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
* merge API tables to existing dataset owner and schema field table
* Add confidential and recursive column to dict_dataset_field
2016-08-24 09:10:35 -07:00
Yi (Alan) Wang
078e90e8bd
Add multiproduct and git repo metadata etl job ( #202 )
...
* Add multiproduct and git repo metadata etl job
* implement the dataset availability section
* Extract commit hash use it when querying acl
* Use FileWriter to write records into CSV file
* Remove unnecessary log entries from kafka processor
* Fix the incompatibility between integer repo_id in db and string field in record
2016-08-12 12:26:55 -07:00
Eric Sun
67479eac58
log_dataset_instance_load_status
2016-08-01 13:00:39 -07:00
Eric Sun
50c222d478
Avro schema for Kafka topics
2016-07-28 16:55:47 -07:00
Eric Sun
f745642212
add datacenter, deploymenttier, cluster to describe dataset instance
2016-07-28 16:38:03 -07:00
Yi Wang
6d4706bc62
Ingest Gobblin tracking events into wherehows using Kafka consumer client
2016-07-25 15:03:29 -07:00
jbai
f3c299480f
update the column names from schema to schema_text and view_expanded_text to ddl_text
2016-07-20 18:01:25 -07:00
jbai
33b05cde4b
tracking the dalids schema and expanded text by versions
2016-07-20 15:59:11 -07:00
jbai
c1b96db103
make the column version_sort_id of table dict_dataset_instance it not null and default value is 0
2016-06-29 23:00:18 -07:00
jbai
9166db7563
update the dict_dataset_instance data loading sql since table key changed
2016-06-29 18:00:10 -07:00
Eric Sun
807e59330e
travis mysql setup
2016-06-22 21:21:19 -07:00
Eric Sun
56cebd0d9c
add dict_dataset_instance and cfg_object_name_map to track dataset on multiple clusters and their replication dependency and view dependency
2016-06-22 19:40:05 -07:00
jerrybai2009
8a9eeb1bb8
Merge pull request #126 from jerrybai2009/master
...
support the elasticsearch as search engine
2016-05-23 18:12:55 -07:00
jbai
a2e42d60f3
add the elasticsearch index build and update file
2016-05-23 17:58:37 -07:00
Rafal Kluszczynski
8501b1d7b8
fix: update DDL for job_execution_ext_reference tables to work with mysql 5.7.x
2016-05-23 10:37:30 +02:00
Eric Sun
5ff4d35f51
add cfg_object_name_map and job_execution_ext_reference
2016-05-19 12:39:06 -07:00
SunZhaonan
31de21ddcf
pass parameter through file.
2016-05-03 16:25:56 -07:00
SunZhaonan
b202832741
Innodb engine DDL. Add config for timeout and load sample.
2016-04-05 12:43:02 -07:00
SunZhaonan
6077f6565b
Fix DDL db_type_id default to 0
2016-04-04 13:40:03 -07:00
SunZhaonan
c4671d2579
Add field comments ETL
...
Fix API bug of tech_matrix_id
Add key in comment table
2016-03-14 14:23:33 -07:00
SunZhaonan
5e9ae37952
Change to multi processing instead of multi thread. Fix hive ETL bug
2016-02-29 16:37:03 -08:00
Eric Sun
f183d8e6da
clean up: dict_business_metric (product_page_key_group_sk => sub_category)
2016-02-29 00:13:40 -08:00
jbai
bae310e715
add the license header to fix the build error
2016-02-25 11:26:57 -08:00
Eric Sun
470a9f8984
Create readme.MD
2016-02-16 16:36:47 -08:00
Eric Sun
7b334e89f1
Create create_all_tables_wrapper.sql
2016-02-16 16:31:29 -08:00
Eric Sun
7971d5b692
Update users.sql
2016-02-16 00:41:07 -08:00
SunZhaonan
f5f3871c94
Fix UI bugs of duplicate node and urn null, urn lowercase. Add missing DDL.
2016-02-12 11:59:14 -08:00
SunZhaonan
c3da00003e
Fix bugs. Reenforce logging. Format jython scripts. Add missing table DDL.
2016-02-03 19:22:18 -08:00
Zhen Chen
7fca60a527
add ref flow id for sub flow jobs
2015-12-17 16:26:15 -08:00
Zhen Chen
d6f77a6f06
fix ldap org hierachy is empty bug and minor changes
2015-12-16 17:01:55 -08:00
SunZhaonan
cd44daba5d
Merge with master
2015-12-16 15:54:50 -08:00
