335 Commits

Author SHA1 Message Date
david-leifker
b71baac0b8
logging(auth): adding optional logging to authentication exceptions (#7929) 2023-05-01 13:18:05 -05:00
david-leifker
9ef281acf2
fix(system-update): fix no wait flag (#7927) 2023-04-30 08:12:11 -05:00
Pedro Silva
9199c89f13
fix(cache): Fix cache key serialization in search service (#7858)
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-04-20 19:07:23 +01:00
RyanHolstien
cbf0809c4c
chore(snakeyaml): upgrade to snakeyaml 2 (#7786)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-04-13 12:01:51 -05:00
david-leifker
82e55636a7
feat(search): allow longer customProperties (#7804) 2023-04-12 16:59:11 -05:00
RyanHolstien
8ffb6a3585
feat(config): allow hooks to be enabled/disabled (#7761) 2023-04-07 11:10:05 -05:00
siladitya
4336252cb7
fix(datahub-upgrade): custom timeseries aspect index creation issue. (#7622)
Co-authored-by: si-chakraborty <si.chakraborty@adevinta.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-03-21 09:36:06 -07:00
Shirshanka Das
fc9ad49821
fix(platform): Ensure time based lineage handles noop changes (#7657)
Co-authored-by: aditya-radhakrishnan <aditya@acryl.io>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2023-03-21 10:43:56 -05:00
david-leifker
697e8e2647
fix(misc): misc fixes (#7633) 2023-03-21 19:42:50 +05:30
Indy Prentice
7914e15ed4
fix(logging): increase log level for system-upgrade job to complete before starting (#7566)
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
2023-03-14 11:38:04 -05:00
david-leifker
72198f98c0
feat(search): searchAcrossEntities/Lineage improvements (#7550)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-03-12 18:18:51 -07:00
John Joyce
947990ac8b
fix(): Fix Query Detail Modal Scroll + add misc log messages (#7530) 2023-03-09 18:02:13 -08:00
RyanHolstien
4e250cbd51
feat(elasticsearch): support searchAfter (#7235) 2023-02-22 19:03:14 -06:00
RyanHolstien
98f3f93442
feat(auth): REST API authorization (#6614) 2023-02-22 15:45:45 -06:00
Pedro Silva
4f8a5c3800
feat(timeBasedLineage): add feature flag for always producing MCL (#7407) 2023-02-22 15:52:51 +00:00
John Joyce
08a215951c
feat(queries): Overhaul Queries Tab (#7366) 2023-02-20 11:10:18 -08:00
RyanHolstien
bc61d1e446
fix(homepage): make entity counts execute in parallel and make cache configurable (#7249) 2023-02-14 07:42:39 -06:00
Aseem Bansal
a60d27b40f
fix(bootstrap): do not re-run retention policy ingestion (#7295) 2023-02-10 20:56:19 +05:30
david-leifker
b4b3a39829
fix(system-update): fixes system-update with more than 1 partition (#7302) 2023-02-09 18:54:20 -06:00
vojtechneradatos
c72d0105c8
feat(api): Timeline API supports Glossary Terms now (#7229)
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-02-08 14:52:29 -06:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements (#6894) 2023-01-31 18:44:37 -06:00
RyanHolstien
8323128437
feat(cache): add hazelcast distributed cache option (#6645)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-01-19 09:03:08 -08:00
david-leifker
6d9da9b5e9
fix(secret-service): fix default encrypt key (#7074) 2023-01-18 17:56:45 -06:00
david-leifker
6a0c96ac26
fix(config): adding quotes (#7040) 2023-01-13 13:55:10 -06:00
david-leifker
55a5e68c9a
fix(config): adding quotes (#7038) 2023-01-13 12:43:45 -06:00
Gabe Lyons
5cecc3c48e
feat(queries): [Experimental] Allow customization of # of queries in Query tab via env var (#6964) 2023-01-06 10:54:45 -08:00
david-leifker
cb6314c51d
fix(elasticsearch): fixes out of order runId writes (#6845)
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-27 17:50:11 -06:00
Aseem Bansal
bef87abd6c
refactor(graphql): make graphqlengine easier to use (#6865) 2022-12-27 22:58:12 +05:30
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) (#6744)
* fix(security): commons-text in frontend

* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint

* testing docker build

* Add retry configuration options for entity client

* Kafka debugging

* fix(kafka-setup): parallelize topic creation

* Adjust docker build

* Docker build updates

* WIP

* fix(lint): metadata-ingestion lint

* fix(gradle-docker): fix docker frontend dep

* fix(elastic): fix race condition between gms and mae for index creation

* Revert "fix(elastic): fix race condition between gms and mae for index creation"

This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.

* fix(test): fix datahub frontend test for clean/test cycle

* fix(test): datahub-frontend missing assets in test

* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer

* gitingore update

* fix(docker): remove platform on docker base image, set by buildx

* refactor(kafka-producer): update kafka producer tracking/logging

* updates per PR feedback

* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool

Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
djordje-mijatovic
e6c48e5f19
feat(kafka): expose default kafka producer mechanism (#6381)
* Expose Kafka Sender Retry Parameters

* Implement KafkaHealthChecker

* feat(kafka): expose default kafka producer mechanism
2022-12-20 14:41:24 -06:00
Chris Collins
f559dcc282
feat(lineage) Adds ability to edit lineage manually from the UI (#6816) 2022-12-20 10:54:08 -05:00
RyanHolstien
f85fd157e9
fix(elasticsearch): build in resilience against IO exceptions on httpclient (#6680)
* fix(elasticsearch): build in resilience against IO exceptions on http client
2022-12-13 18:11:02 -06:00
John Joyce
54230a8d81
feat(): Views Feature Milestone 1 (#6666) 2022-12-12 12:30:09 -08:00
david-leifker
be9348b518
fix(build): rename conflicting module auth-api (#6728)
* fix(build): rename conflicting module `auth-api`
2022-12-09 18:06:51 -06:00
mohdsiddique
f2a83f3074
feat(gms): Pluggable Authentication & Authorization Framework (#6634) 2022-12-06 10:22:41 -08:00
syedzoherer
ffea9302d9
Add AWS Postgres Iam Auth jar to GMS (#6371)
* Add AWS Postgres Iam Auth jar to GMS

* Added IAM support for Postgres

* Setting IAM support for Postgres as false by default

* format fix

Co-authored-by: syed.javed <syed.javed@precisely.com>
2022-12-05 13:07:26 -06:00
david-leifker
2de9d3d5bf
fix(logging): Remove lombok as source of slf4j-api, convert to compileOnly where possible (#6616) 2022-12-04 19:57:47 -08:00
syedzoherer
5010e0cf69
Added I AM auth for Opensearch (#6370)
* Added I AM auth for Opensearch

* Code refactor and removed default Region for AWS cred.

* Added missing opensearch region exception

* Update RestHighLevelClientFactory.java

* Fixed stylecheck

* Update RestHighLevelClientFactory.java

* Update RestHighLevelClientFactory.java

Co-authored-by: syed.javed <syed.javed@precisely.com>
2022-11-22 10:49:28 -06:00
david-leifker
33fd876421
feat(elasticsearch): Updates to elasticsearch configuration, dao, and tests (#6269) 2022-11-15 18:03:11 -08:00
Chris Collins
e8aaf2a337
feat(ui) Add upgrade step to enable CLL impact analysis for existing data (#6427) 2022-11-15 16:45:33 -05:00
Corentin
fbcfcfc147
feat(recommendations): add last edited entities (#6329) 2022-11-14 22:15:48 -08:00
Aditya Radhakrishnan
2a4ec99d89
fix(change-event): remove unnecessary dependencies on EntityChangeEventGeneratorRegistryFactory (#6431) 2022-11-14 12:40:48 -08:00
Aditya Radhakrishnan
873b100a47
refactor(change-events): refactor AspectDiffers to EntityChangeEventGenerators (#6320)
* refactor(change-events): refactor AspectDiffers to EntityChangeEventGenerators

* feat(change-event): add change events for AssertionRunEvent

* feat(change-event): add change events for DataProcessInstanceRunEvent
2022-11-01 09:53:43 -07:00
Aditya Radhakrishnan
e8259788a3
feat(tracking): add telemetry for frontend events (#6129) 2022-10-06 18:56:32 -07:00
Aditya Radhakrishnan
ee1a1ef45b
feat(posts): add posts feature to DataHub (#6110) 2022-10-04 15:37:28 -07:00
RyanHolstien
bfb903cfb8
feat(ingest): add async option to ingest proposal endpoint (#6097)
* feat(ingest): add async option to ingest proposal endpoint

* small tweak to validate before write to K, also keep existing path for timeseries aspects

* avoid double convert

Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-03 19:56:19 -05:00
Shirshanka Das
48b46971b8
feat(ingest): remove hardcoded env variable default for cli version (#6075) 2022-10-02 16:46:34 -07:00
Pedro Silva
164bc1aa53
Revert "chore(setup): change defaults for partitions (#6074)" (#6086) 2022-09-29 15:44:32 +05:30
Aseem Bansal
4cf635e16e
chore(setup): change defaults for partitions (#6074) 2022-09-28 15:04:35 +05:30
Aseem Bansal
1c4e127ff9
feat(retention): disable applying retention on bootstrap (#6066) 2022-09-27 12:24:37 -07:00