397 Commits

Author SHA1 Message Date
Gabe Lyons
febbced383
marking data process aspects as deprecated (#3172) 2021-09-28 16:43:44 -07:00
John Joyce
9f4906ee3f
feat(UI): UI Migration for Charts, Dashboards, Pipelines, Tasks + Glossary Terms and Links for all. (#3303) 2021-09-28 10:30:37 -07:00
Gabe Lyons
ff527f4bed
feat(foreign keys): add foreign key models (#3275) 2021-09-22 10:29:27 -07:00
rslanka
c418bc845c
feat(Analytics): Support for Timeseries Aggregated Statistics (#3207)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Dexter Lee <dexter@acryl.io>
2021-09-14 18:35:10 -07:00
Gabe Lyons
2de4ea9566
feat(business glossary): Add support to add & remove glossary terms from the UI (#3213) 2021-09-13 09:16:37 -07:00
Dexter Lee
e30d7238c0
feat(search): Add searchable annotation to maps (#3136) 2021-09-07 22:58:44 -07:00
John Joyce
ccb09a6966
feat(access control): Fine-Grained Access Control M1 (#3182) 2021-09-02 19:05:13 -07:00
Swaroop Jagadish
b673f1a52a
fix(frontend): fixing broken link to default avatar (#3154) 2021-08-24 17:24:48 -07:00
John Joyce
81eb4f19bf
feat(sso): Just-In-Time User & Group Provisioning on SSO Login (oidc) (#3082) 2021-08-20 07:42:18 -07:00
John Joyce
f40bf1ce19
feat(group ui): Basic group search membership in UI (#3094) 2021-08-16 20:47:18 -07:00
Shikha-Trivedi-Saxo
c9c1ba457e
feat(business-glossary): Glossary term relationship backend (#3012) 2021-08-15 23:05:03 -07:00
Lal Rishav
9cbc147975
feat(business-glossary): added field level glossary terms (#3048) 2021-08-15 21:59:12 -07:00
John Joyce
8c9c696cdd
feat(ingest): Adding an Okta Integration to extract Users, Groups, Group Membership (#3043) 2021-08-11 18:49:16 -07:00
rslanka
8844240328
feat: Adding support for nested schemas in ingestion and visualization (#3079) 2021-08-11 15:47:18 -07:00
Lal Rishav
94b35fe8a0
fix(Business Glossary): updated glossary term search strategy (#2973) 2021-08-05 23:31:06 -07:00
Gabe Lyons
8c4a1414fc
fix(gms): handling partial system metadata in gms (#3030) 2021-08-04 12:26:29 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect (#2983)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Dexter Lee
15c0c4dfb3
fix(browse): Fix browse pagination and multi-browse path issue (#2984) 2021-07-30 10:48:32 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960) 2021-07-29 20:04:40 -07:00
Kevin Hu
43e61d9628
feat(models): remove versions from metrics and hyperparams (#2938) 2021-07-22 22:18:45 -07:00
Kevin Hu
6dbc59940a
feat(ingest): refactor mlModel grouping and add browsepaths (#2929) 2021-07-22 13:33:15 -07:00
Lal Rishav
99790d9052
fix(react): enable relation between glossary term and datasets searchable (#2806)
* 2805 | glossary term linked to dataset is not displayed
2021-07-22 07:25:29 -07:00
Kevin Hu
736249f0c7
feat(ingest): extract SageMaker metrics, hyperparameters, and external URLs (#2910) 2021-07-21 21:30:07 -07:00
Kevin Hu
6abd5e191a
feat(ingest): lineage for SageMaker model endpoints and groups (#2894) 2021-07-19 11:30:43 -07:00
Lal Rishav
ae014bf2e1
fix(glossary):default browse path for glossary term (#2788) 2021-07-16 09:15:24 -07:00
Kevin Hu
44ed2f3684
feat(ingest): extract lineage between SageMaker jobs and models (#2868) 2021-07-15 18:56:13 -07:00
Kevin Hu
a2106ca9e8
feat(ingest): SageMaker jobs and models (#2830) 2021-07-08 16:16:16 -07:00
Harshal Sheth
2f921d15e8
fix(ingest): avoid setting timestamps unless source system provides it (#2843) 2021-07-08 12:11:06 -07:00
Kevin Hu
4da76726d3
feat(ingest): SageMaker feature store ingestion (#2758) 2021-06-29 19:43:31 -07:00
Dexter Lee
ae8a8893b0
fix(nocode): removing service PDL (#2772) 2021-06-25 10:58:10 -07:00
Harshal Sheth
19b2a42a00
feat: usage stats (part 2) (#2762)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-06-24 19:44:59 -07:00
Harshal Sheth
937f02c6bc
feat: usage stats (part 1) (#2750)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-06-24 17:11:00 -07:00
Kevin Hu
a89094da5b
feat(ingest): add support for Glue ETL jobs (#2687) 2021-06-22 11:33:22 -07:00
Gabe Lyons
0750332714
fix(editable descriptions): adding indexing for editable descriptions (#2710) 2021-06-17 10:55:26 -07:00
Brian
a5f9b8dfe9
feat(entities): add markdown description update/viewer feature in dataset, datajob, dataflow, chart and dashboard, update ui/ux (#2707) 2021-06-16 15:48:27 -07:00
Gabe Lyons
523c3bf1d4
feat(aspects): support fetching of versioned aspects (#2677) 2021-06-16 10:03:21 -07:00
Gabe Lyons
0b75b4a96a
removing whitespace from service aspect (#2688) 2021-06-12 17:48:36 -07:00
Gabe Lyons
28b5856c36
fix(no-code): Adding Chart input relationship annotations (#2669)
* chart input relationship pdl fix

* commiting schema.avsc changes
2021-06-09 15:09:50 -07:00
Kevin Hu
ebdaa0e359
feat(ingest): Feast ingestion integration (#2605)
* Add feast testing setup

* Init Feast test script

* Add feast to dependencies

* Update feast descriptors

* Sort integrations

* Working feast pytest

* Clean up feast docker-compose file

* Expand Feast tests

* Setup feast classes

* Add continuous and bytes data to feature types

* Update field type mapping

* Add PDLs

* Add MLFeatureSetUrn.java

* Comment out feast setup

* Add snapshot file and update inits

* Init Feast golden files generation

* Clean up Feast ingest

* Feast testing comments

* Yield Feature snapshots

* Fix Feature URN naming

* Update feast MCE

* Update Feature URN prefix

* Add MLEntity

* Update golden files with entities

* Specify feast sources

* Add feast source configs

* Working feast docker ingestion

* List entities and features before adding tables

* Add featureset names

* Remove unused

* Rename feast image

* Update README

* Add env to feast URNs

* Fix URN naming

* Remove redundant URN names

* Fix enum backcompatibility

* Move feast testing to docker

* Move URN generators to mce_builder

* Add source for features

* Switch TypeClass -> enum_type

* Rename source -> sourceDataset

* Add local Feast ingest image builds

* Rename Entity -> MLPrimaryKey

* Restore features and keys for each featureset

* Do not json encode source configs

* Remove old source properties from feature sets

* Regenerate golden file

* Fix race condition with Feast tests

* Exclude unknown source

* Update feature datatype enum

* Update README and fix typos

* Fix Entity typo

* Fix path to local docker image

* Specify feast config and version

* Fix feast env variables

* PR fixes

* Refactor feast ingest constants

* Make feature sources optional for back-compatibility

* Remove unused GCP files

* adding docker publish workflow

* Simplify name+namespace in PrimaryKeys

* adding docker publish workflow

* debug

* final attempt

* final final attempt

* final final final commit

* Switch to published ingestion image

* Update name and namespace in java files

* Rename FeatureSet -> FeatureTable

* Regenerate codegen

* Fix initial generation errors

* Update snapshot jsons

* Regenerated schemas

* Fix URN formats

* Revise builds

* Clean up feast URN builders

* Fix naming typos

* Fix Feature Set -> Feature Table

* Fix comments

* PR fixes

* All you need is Urn

* Regenerate snapshots and update validation

* Add UNKNOWN data type

* URNs for source types

* Add note on docker requirement

* Fix typo

* Reorder aspect unions

* Refactor feast ingest functions

* Update snapshot jsons

* Rebuild

Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-09 15:07:04 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling (#2629)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-03 13:24:33 -07:00
Brian
aa8ba1b8e9
feat(dataflow): update dataflow to have datajobs in new tab (#2579) 2021-05-18 21:25:42 -07:00
John Joyce
262108e3aa
fix(model): removing reference to go link in SchemaFieldPath model (#2530) 2021-05-11 15:03:04 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields (#2228)
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
2021-05-10 13:20:23 -07:00
Dexter Lee
b4457afe30
feat(search): Add search for field level description and tags (#2491) 2021-05-05 14:04:02 -07:00
Gabe Lyons
851e00ba9f
feat(lineage): implement support for datasets, charts and dashboards downstream lineage fetching in a generic way (#2397)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Brian <brianwebtek@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2021-04-23 00:18:39 -07:00
Gabe Lyons
c7b49de67b
feat(ingest): adding superset ingestion source (#2425) 2021-04-22 00:11:54 -07:00
Fredrik Sannholm
e88a671959
Fix(search): fix datajob and dataflow search mappings (#2418) 2021-04-21 12:04:20 -07:00
Gabe Lyons
039fe597f7
feat(tags): editing tags from react client on datasets, schemas, charts & dashboards (#2248) 2021-03-18 11:52:14 -07:00
Harshal Sheth
b8462028c3
feat(ingest): various minor fixes (#2246) 2021-03-17 23:05:05 -07:00
Dexter Lee
c362cc4388
feat(tags): Enable search for datasets by tags (#2240) 2021-03-15 22:37:06 -07:00