Shirshanka Das
b3c790aab6
feat: Add support for Data Products ( #8039 )
...
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2023-05-17 07:17:25 +00:00
John Joyce
b46822399c
feat(timeseries): Support sorting timeseries aspects by non-timestampMillis field + fix operations resolver ( #7840 )
2023-04-18 09:10:04 -07:00
Shirshanka Das
fc9ad49821
fix(platform): Ensure time based lineage handles noop changes ( #7657 )
...
Co-authored-by: aditya-radhakrishnan <aditya@acryl.io>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2023-03-21 10:43:56 -05:00
RyanHolstien
4e250cbd51
feat(elasticsearch): support searchAfter ( #7235 )
2023-02-22 19:03:14 -06:00
RyanHolstien
98f3f93442
feat(auth): REST API authorization ( #6614 )
2023-02-22 15:45:45 -06:00
John Joyce
08a215951c
feat(queries): Overhaul Queries Tab ( #7366 )
2023-02-20 11:10:18 -08:00
Aseem Bansal
a60d27b40f
fix(bootstrap): do not re-run retention policy ingestion ( #7295 )
2023-02-10 20:56:19 +05:30
Sergio Gómez Villamor
2f2eda0951
feat(graphql): enabling graphql for data platform instance aspects ( #7177 )
2023-02-08 14:56:40 -05:00
Nate Bryant
dbc89c8680
style(models): Replaces non-ASCII charactes in pdl files with ASCII c… ( #7105 )
2023-01-23 18:46:03 -08:00
RyanHolstien
27fe7eb82c
fix(uri): marks uri field as deprecated, removes problem code, and adds coercer for usages of URI typeref ( #7093 )
2023-01-20 14:32:02 -06:00
John Joyce
22ee828236
feat(backend): Supporting Embeddable Previews for Dashboards, Charts, Datasets ( #6875 )
2023-01-03 15:40:39 -08:00
Chris Collins
f559dcc282
feat(lineage) Adds ability to edit lineage manually from the UI ( #6816 )
2022-12-20 10:54:08 -05:00
John Joyce
54230a8d81
feat(): Views Feature Milestone 1 ( #6666 )
2022-12-12 12:30:09 -08:00
John Joyce
bbcec3b226
refactor(ui): Adding caching for users, groups, and roles ( #6673 )
2022-12-08 09:23:15 -08:00
Aditya Radhakrishnan
f0f0355a83
feat(onboarding): adds framework and some steps for onboarding steps UI ( #6462 )
...
* feat(onboarding): adds models and API for onboarding steps feature
* feat(onboarding): adds backend for onboarding steps feature
* feat(onboarding): adds framework and some steps for onboarding steps UI
2022-12-07 16:21:55 -08:00
John Joyce
d1b12243bb
feat(models): Adding 'created', 'lastModified' timestamp to Dataset, Container, Dashboard, Chart ( #6527 )
2022-11-28 18:18:34 -08:00
John Joyce
a400eb0d52
refactor(ui): Adding apollo caching to manage domains page. ( #6494 )
2022-11-23 12:31:31 -08:00
Aditya Radhakrishnan
873b100a47
refactor(change-events): refactor AspectDiffers to EntityChangeEventGenerators ( #6320 )
...
* refactor(change-events): refactor AspectDiffers to EntityChangeEventGenerators
* feat(change-event): add change events for AssertionRunEvent
* feat(change-event): add change events for DataProcessInstanceRunEvent
2022-11-01 09:53:43 -07:00
Chris Collins
cd1331fffe
feat(lineage) Add column-level impact analysis feature ( #6272 )
2022-10-26 16:43:39 -04:00
Aditya Radhakrishnan
e8259788a3
feat(tracking): add telemetry for frontend events ( #6129 )
2022-10-06 18:56:32 -07:00
Aditya Radhakrishnan
ee1a1ef45b
feat(posts): add posts feature to DataHub ( #6110 )
2022-10-04 15:37:28 -07:00
Aditya Radhakrishnan
325b959ea6
feat(roles): add ability to invite users into a role ( #6015 )
2022-09-23 16:48:23 -07:00
RyanHolstien
0728656fbf
feat(patch): initial support of json patch style semantics in MCPs ( #5901 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-22 23:52:18 -07:00
Gabe Lyons
c41f92c8d0
feat(model): Add optional message field to auditstamp ( #5611 )
...
* adding messge to audit stamp
* Update AuditStamp.pdl
* Update AuditStamp.pdl
2022-08-11 13:48:49 -07:00
RyanHolstien
f847fa31c9
fix(idea): change location of coercer to make intellij not complain about classes ( #4918 )
2022-05-13 16:19:53 -07:00
RyanHolstien
65fb49f4b4
feat(versionedDataset): adds a versionStamp to timeline response & adds versionStamp param to dataset graphql ( #4727 )
2022-04-29 12:04:38 -07:00
Xu Wang
34b36c0fe1
feat(notebook): graphqul related logic change for notebook ( #4468 )
...
* feat(notebook) graphqul related logic change for notebook
* resolve pr comments
1. add dataPlatformInstance aspect support
2. make NotebookCell required
* resolve :datahub-graphql-core:checkstyleMain failure
Co-authored-by: Xu Wang <xu.wang@grandrounds.com>
2022-03-28 13:52:41 -07:00
Sergio Gómez Villamor
a2a939d406
feat(model): adds PRE in the FabricType enum ( #4226 )
2022-03-01 16:45:41 -08:00
Ravindra Lanka
232f7c5b21
feat(model): Column level lineage model ( #4248 )
...
co-authored-by: ksrinath <k.srinath@gmail.com>
2022-02-24 14:36:19 -08:00
Aditya Radhakrishnan
b5ff558f03
feat(model) - Update FabricType enum to represent more fabrics ( #3997 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-30 18:19:03 -06:00
John Joyce
720311ff2e
Migrating Datasets, Charts, Dashboards, Jobs, Flows to Entity V2 endpoint ( #3897 )
2022-01-14 17:00:17 -08:00
Dexter Lee
45e962d87e
fix(platform): fix extraction for unions to avoid ( #3892 )
2022-01-14 12:20:51 -08:00
RyanHolstien
34c27f076b
feat(removeGMA): remove all dependencies on gma libraries ( #3835 )
2022-01-05 17:32:31 -08:00
John Joyce
2aae2fd601
fix(gms): Adding Rest.li Write-Time Model Validation ( #2999 )
2021-08-03 09:54:13 -07:00
Harshal Sheth
2f921d15e8
fix(ingest): avoid setting timestamps unless source system provides it ( #2843 )
2021-07-08 12:11:06 -07:00
Gabe Lyons
938dbc8871
Revert "fix(gms): add rest.li validation in gms ( #2745 )" ( #2754 )
...
This reverts commit f0b4adc0e498ad5e549221494334befc1964b1dc.
2021-06-23 23:27:47 -07:00
John Joyce
f0b4adc0e4
fix(gms): add rest.li validation in gms ( #2745 )
2021-06-23 13:48:11 -07:00
Kevin Hu
ebdaa0e359
feat(ingest): Feast ingestion integration ( #2605 )
...
* Add feast testing setup
* Init Feast test script
* Add feast to dependencies
* Update feast descriptors
* Sort integrations
* Working feast pytest
* Clean up feast docker-compose file
* Expand Feast tests
* Setup feast classes
* Add continuous and bytes data to feature types
* Update field type mapping
* Add PDLs
* Add MLFeatureSetUrn.java
* Comment out feast setup
* Add snapshot file and update inits
* Init Feast golden files generation
* Clean up Feast ingest
* Feast testing comments
* Yield Feature snapshots
* Fix Feature URN naming
* Update feast MCE
* Update Feature URN prefix
* Add MLEntity
* Update golden files with entities
* Specify feast sources
* Add feast source configs
* Working feast docker ingestion
* List entities and features before adding tables
* Add featureset names
* Remove unused
* Rename feast image
* Update README
* Add env to feast URNs
* Fix URN naming
* Remove redundant URN names
* Fix enum backcompatibility
* Move feast testing to docker
* Move URN generators to mce_builder
* Add source for features
* Switch TypeClass -> enum_type
* Rename source -> sourceDataset
* Add local Feast ingest image builds
* Rename Entity -> MLPrimaryKey
* Restore features and keys for each featureset
* Do not json encode source configs
* Remove old source properties from feature sets
* Regenerate golden file
* Fix race condition with Feast tests
* Exclude unknown source
* Update feature datatype enum
* Update README and fix typos
* Fix Entity typo
* Fix path to local docker image
* Specify feast config and version
* Fix feast env variables
* PR fixes
* Refactor feast ingest constants
* Make feature sources optional for back-compatibility
* Remove unused GCP files
* adding docker publish workflow
* Simplify name+namespace in PrimaryKeys
* adding docker publish workflow
* debug
* final attempt
* final final attempt
* final final final commit
* Switch to published ingestion image
* Update name and namespace in java files
* Rename FeatureSet -> FeatureTable
* Regenerate codegen
* Fix initial generation errors
* Update snapshot jsons
* Regenerated schemas
* Fix URN formats
* Revise builds
* Clean up feast URN builders
* Fix naming typos
* Fix Feature Set -> Feature Table
* Fix comments
* PR fixes
* All you need is Urn
* Regenerate snapshots and update validation
* Add UNKNOWN data type
* URNs for source types
* Add note on docker requirement
* Fix typo
* Reorder aspect unions
* Refactor feast ingest functions
* Update snapshot jsons
* Rebuild
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-09 15:07:04 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling ( #2629 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-03 13:24:33 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields ( #2228 )
...
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
2021-05-10 13:20:23 -07:00
Gabe Lyons
adfe60e97a
feat(tags): adding support for read/write of tags in gms & read-only in react datahub-frontend. ( #2164 )
2021-03-07 11:26:47 -08:00
RyanHolstien
ea86ade29b
feat: ML Model Backend Implementation ( #1896 )
...
Co-authored-by: RyanHolstien <rholstien@expediagroup.com>
2021-02-17 13:28:13 -08:00
Harsh Shah
06cca77a6b
feat(models): Add DataFlow and DataJob models ( #2008 )
...
Add DataFlow and DataJob urn and aspect models.
The aspects include metadata associated with Data processing flow and job entities,
associated inputs and outputs of a job.
2020-11-30 17:48:29 -08:00
Kerem Sahin
4d8320e4a0
feat(dashboard): Dashboards backend implementation ( #1884 )
2020-11-23 09:25:58 -08:00
Kerem Sahin
733893f5f9
feat(dashboard): Dashboard models update ( #1932 )
...
* feat(dashboard): Dashboard models update
* Keep chartId/dashboardId fields in the URN definitions and add fields for chartURL/dashboardURL into info aspects
* Rebase and address some comments
2020-11-12 11:17:22 -08:00
John Plaisted
32e18a1fc7
fix: fix build definition of DatasetFieldUrn ( #1991 )
2020-11-11 16:54:05 -08:00
John Plaisted
60e43061d8
[Breaking] Update to GMA 0.2.0 and fix Urn definitions. ( #1977 )
...
Urn definitions needed to be updated since 0.2.0 changed the base Urn class.
I also added some more urn coercers that were missing.
2020-11-11 16:06:29 -08:00
Nagarjuna Kanamarlapudi
7d574d1094
feat(field-level-lineage): Add models for field level lineage ( #1936 )
...
* feat(field-level-lineage): adding models for field level lineage
adding models for field level lineage. Introduce DatasetFieldUrn as a unique identifier for dataset field
2020-11-09 14:08:48 -08:00
John Plaisted
25b663cc18
refactor: move code to linkedin/datahub-gma. ( #1955 )
...
Move code to linkedin/datahub-gma.
"GMA" (Generalized Metadata Architecture) is the backend of DataHub, and has been moved to its own repository.
This deletes the code that was moved and uses jars that GMA publishes to bintray to load it.
Note that not all of GMA was moved, but most of it. We may still move more things to the other repository in the future.
2020-10-23 15:14:57 -07:00
John Plaisted
230ecce9d6
Revert "fix: validate entity type for an urn ( #1958 )" ( #1959 )
...
This change should've been made in datahub-gma instead.
This reverts commit 127d84e3f9bc8b0638df535a594f21d8e34a508b.
2020-10-23 14:33:11 -07:00
