RyanHolstien
848574dbff
fix(platform): patch for entity creation, honor async flag on request ( #6504 )
2022-11-22 11:05:09 -08:00
Chris Collins
970e9f0aa7
fix(ui) Fix lineage graph rendering with duplicate nodes ( #6480 )
2022-11-18 12:28:30 -05:00
david-leifker
33fd876421
feat(elasticsearch): Updates to elasticsearch configuration, dao, and tests ( #6269 )
2022-11-15 18:03:11 -08:00
djordje-mijatovic
ca67e79b3b
feat(neo4j): Neo4j multihop support ( #6104 )
2022-11-15 09:14:01 -08:00
Chris Collins
31f90a4b52
fix(ui) Fix two lineage visualization bugs ( #6278 )
2022-10-26 13:56:32 -04:00
david-leifker
7cdc9f7fcf
test(neo4j): Improve test performance ( #6142 )
...
Reuse neo4j graph service in test for performance improvements.
2022-10-24 08:49:06 -07:00
Gabe Lyons
ce90310dd0
feat(advanced-search): Complete Advanced Search: backend changes & tying UI together ( #6068 )
...
* stashing progress
* adding remove option
* more progress
* editing
* further in
* additional rendering improvements
* stashing adv search progress
* stashing more progress
* propagating not filters back to UI
* more frontend progress
* more filters working
* getting ready for data platform selector
* add platform select functionality
* locking out switching btwn advanced and standard filters
* final polish
* remove unneeded code
* added unit and cypress tests
* resolutions after merge
* adding documentation
* cleaning up & refactoring
* removing console.log
* minor ui fix & removing unneeded code
* fixing lineage search
* fixing lints
* fix display of degree
* fixing test
* fixing lint
* responding to comments
* fixing tests
* fix smoke tests
* fixing cypress
* fixing cypress test
* responding to comments
2022-10-04 10:20:04 -07:00
RyanHolstien
bfb903cfb8
feat(ingest): add async option to ingest proposal endpoint ( #6097 )
...
* feat(ingest): add async option to ingest proposal endpoint
* small tweak to validate before write to K, also keep existing path for timeseries aspects
* avoid double convert
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-03 19:56:19 -05:00
Shirshanka Das
e688dd554a
chore(deps): upgrade neo4j to 4.4.x ( #6101 )
2022-10-02 23:16:45 -07:00
RyanHolstien
0728656fbf
feat(patch): initial support of json patch style semantics in MCPs ( #5901 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-22 23:52:18 -07:00
RyanHolstien
2c659214ab
feat(gms): Write back lineage search results to in-memory cache bound to feature flag ( #6006 )
...
Co-authored-by: Piyush Narang <piyushn@stripe.com>
2022-09-22 15:01:24 -07:00
Parham Ghazanfari
2712dd1903
fix(patch-entity-registry): Use AspectSpec to retrieve aspect class in order to support custom entity ingestion via patch mechanism. ( #5936 )
...
* fix(patch-entity-registry): Remove exception for entities with key aspects.
* test(patch-entity-registry): Fix failing unit test for entities with key aspect
* test(patch-entity-registry): Fix checkstyle for PatchEntityRegistryTest.java
* fix(patch-entity-registry): Remove duplicate semicolon.
* fix(patch-entity-registry): Use AspectSpec to retrieve aspect class in order to support custom entity ingestion via patch mechanism.
* fix(patch-entity-registry): Fixed incorrect merge conflict resolution in BrowsePathUtils.java
* fix(patch-entity-registry): Remove extraneous import from BrowsePathUtils.java
Co-authored-by: Parham Ghazanfari <pghazanfari53@yahoo.com>
2022-09-19 10:47:20 -05:00
John Joyce
38f111117c
refactor(gms): Adding Java Entity Services ( #5931 )
2022-09-14 10:19:13 -07:00
Justin Marozas
133b5e0348
feat(search): Add support for Elasticsearch object field type ( #5891 )
...
* object-type searchable field WIP
* elasticsearch object-type field WIP
* talk about the object fieldType in the docs
2022-09-14 11:50:19 -05:00
Ravindra Lanka
386719f15a
feat(platform): timeseries - Server & Client side changes to support timeseries aspect deletion & rollback. ( #4756 )
2022-09-11 11:27:46 -07:00
John Joyce
d15518f0d9
fix(browse): Fixing browse path to remove requirement for simple name suffix ( #5634 )
2022-09-07 13:32:38 -07:00
Gabe Lyons
2e332d8c09
fix(graph service): only query for entities that should have lineage [Breaking Change] ( #5539 )
...
* allow filtering multiple entity types in graph service
* fixing up tests
* fixing imports
* lint fix
* another lint fix
* fixing another lint
* responding to comments
* fixing entity registry test
* always fetch yourself
2022-08-11 13:15:38 -07:00
John Joyce
57b7ade1f0
feat(ui): Add rich UI ingestion run summary ( #5577 )
2022-08-08 14:33:57 -07:00
Justin Marozas
414252351a
fix(cassandra): fix Cassandra queries used by IngestDataPlatformInstancesStep ( #5199 )
2022-07-07 22:20:57 -07:00
John Joyce
b02c7f5952
perf(search): Improve search default performance ( #5311 )
2022-07-01 10:40:51 -07:00
John Joyce
234d27a642
fix(ui): Support deleting references to glossary terms / nodes, users, assertions, and groups ( #5248 )
...
* Adding referential integrity to deletes API
* Updating comments
* Fix build
* fix checkstyle
* Fixing Delete Entity utils Test
2022-06-27 14:31:40 -07:00
Gabe Lyons
baf3f3f33c
feat(dbt): add sibling association logic to associate dbt elements with their target systems ( #5190 )
2022-06-22 08:31:18 -07:00
Chris Collins
2841f32b8e
feat(lineage) Update Lineage tab and Impact Analysis feature ( #5121 )
2022-06-21 07:30:40 -07:00
Dexter Lee
669160a677
feat(run): Create a describe run endpoint for fetching aspects created by the ingestion run ( #4964 )
2022-05-24 14:46:36 -07:00
Justin Marozas
97bce80fc9
refactor(metadata-io): introduce a storage-independent in-memory entity aspect model ( #4957 )
2022-05-24 12:40:42 -07:00
RyanHolstien
c3cfaf8e3c
feat(telemetry): add server side telemetry ( #4925 )
...
Co-authored-by: Kevin Hu <kevinhuwest@gmail.com>
2022-05-17 11:37:45 -07:00
Pedro Silva
78c3ca039e
feat(gms): Add support for deleting reference pointers when deleting by urn ( #4791 )
2022-05-12 19:02:13 -07:00
RyanHolstien
65fb49f4b4
feat(versionedDataset): adds a versionStamp to timeline response & adds versionStamp param to dataset graphql ( #4727 )
2022-04-29 12:04:38 -07:00
Eddie (Xiao Di) Li
01a5b13a7a
feat(Cassandra): Add Cassandra implementation of EntityService ( #3286 )
2022-04-25 17:29:24 -07:00
Dexter Lee
46c15ef0a7
fix(search): Fix urn component settings ( #4672 )
...
* Fix urn component settings
* Add keyword subfield to all fields
* Fix tests
2022-04-14 15:40:03 -07:00
Dexter Lee
55f0412a33
feat(search): Add SearchScore annotation to use fields for search ranking ( #4596 )
...
* Add SearchScore annotation
* Add back test-model
* Remove search features
* Fix to John's comments
* simplify ranker
* Fix checkstyle
2022-04-07 11:07:27 -07:00
Dexter Lee
32324a6476
fix(search): Make sure home page and search pages are consistent ( #4588 )
...
* Centralize entity count cache
* Fix to John's comment
* Fix test
* Fix tests
2022-04-06 22:06:55 -07:00
John Joyce
c69310522b
feat(metadata service): Introducing Platform Events ( #4477 )
2022-03-29 18:32:04 -07:00
RyanHolstien
b843014c5e
Oss/urn validation ( #4511 )
2022-03-29 16:44:58 -07:00
Kevin Neville
d8e6f890a9
fix: Replace old repository link with new link ( #4446 )
2022-03-18 14:12:19 -07:00
Pedro Silva
aa593c32d8
Flexible search on soft delete ( #4405 )
...
* Adds filter logic to correct DB
* Fix build
* Adds documentation & fixes flag typo
* apply review comments
* Adds test for filtered search
* Adds warning log for redundant parameter combo
2022-03-16 16:35:04 -07:00
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default ( #4358 )
...
* Changes rollback behaviour to apply soft deletes by default
Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.
Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.
To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.
* Adds restli specs
* Fixes deleteAspect endpoint & adds support for nested transactions
* Enable regression test & fix docker-compose for local development
* Add generated quickstart
* Fix quickstart generation script
* Adds missing var env to docker-compose-without-neo4j
* Sets status removed=true when ingesting resources
* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands
* Makes elastic search consistent
* Update tests with new behaviour
* apply review comments
* apply review comment
* Forces Elastic search to add documents with status removed false when ingesting
* Reset gradle properties to default
* Fix tests
2022-03-15 12:05:52 -07:00
Dexter Lee
ab0b5162af
fix(lineage): Move lineage registry to entity-registry module ( #4412 )
...
* Move lineage registry
* Fix test
2022-03-15 11:59:59 -07:00
Dexter Lee
a12c9d2a30
Add flag to enable cache ( #4335 )
2022-03-14 16:40:10 -07:00
Dexter Lee
424d357cfd
Fix lineage ( #4326 )
2022-03-07 09:36:49 -08:00
Dexter Lee
18dd5b6c13
feat(Impact Analysis): Support impact analysis to check all downstreams of given entity ( #4322 )
2022-03-04 16:10:25 -08:00
Dexter Lee
daab31d6ef
refactor(extractor): Move extractors to entity-registry ( #4307 )
...
* Move extractors to entity-registry
* Fix test
* Fix checkstyle
2022-03-03 16:57:32 -08:00
Dexter Lee
5d26c86a30
fix(search): Update urn tokenizer to tokenize on periods and slashes ( #4085 )
...
* Add period
* Update urn tokenizer
* Add tests
2022-03-03 10:37:10 -08:00
Ravindra Lanka
1ac20ee764
feat(platform): add timeline api for dataset entity ( #4252 )
2022-02-24 19:40:55 -08:00
HuYueeer
3a0fe44344
feat(search): supporting chinese glossaryterm full text retrieval( #3914 ) ( #3956 )
...
* feat(search): supporting chinese glossaryterm full text retrieval(#3914 )
* refactor(search): modify mainTokenizer to appropriate position(#3914 )
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-24 17:11:43 -08:00
stephenp-gr
fb80e6b7bc
Common ElasticsearchContainer creation ( #4152 )
2022-02-16 16:10:10 -08:00
ksrinath
4c24f386a6
feat(model): data quality model ( #3787 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Mayuri N <mayuri.nehate@gslab.com>
2022-02-02 22:56:08 -08:00
senni0418
80c46f24ac
refactor(search): refactor NUM_RETRIES in esindexbuilder to be configurable ( #3870 )
2022-01-17 15:35:59 -08:00
Dexter Lee
b58609269d
fix(search): Fix issue with filters and autocomplete ( #3868 )
2022-01-12 13:01:36 -08:00
Aseem Bansal
a56f0661d5
perf(ingest): changes to improve ingest performance a bit ( #3837 )
2022-01-06 08:07:16 -08:00
