394 Commits

Author SHA1 Message Date
david-leifker
0a9dc73402
build(docker): metadata-ingestion images build and add slim version (#7412) 2023-03-20 18:06:35 -05:00
Aseem Bansal
c43d7dd9d2
chore(ci): fix flakiness, misc improvements (#7605) 2023-03-17 09:34:00 +05:30
Aseem Bansal
2ef26e8b74
fix(policies): add missing policies, add check to prevent problems (#7586)
Co-authored-by: John Joyce <john@acryl.io>
2023-03-16 09:49:55 +05:30
Aseem Bansal
b2ecd7f968
chore(ci): update dependencies, fix smoke image build (#7580) 2023-03-15 21:07:30 +05:30
Aseem Bansal
39f7f74329
chore(ci): add daylight savings timezone for tests, fix daylight saving bug in analytics charts (#7484) 2023-03-14 16:12:29 +05:30
Aseem Bansal
64b4d0bace
chore(ci): try Qodana Scan for quality (#7560) 2023-03-14 14:26:06 +05:30
Aseem Bansal
1adbc2cab0
chore(ci): upgrade GE version (#7290) 2023-03-02 10:47:38 -08:00
Aseem Bansal
324fc8ee9f
chore(ci): update base dependencies (#7390)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-02-21 16:07:51 -06:00
david-leifker
237933cc8d
build(scan): enable trivy scan ingestion-base (#7389) 2023-02-21 11:15:15 +05:30
Aseem Bansal
f79438ffe9
fix(ci): remove command from cache key as irrelevant for dependency (#7314) 2023-02-13 12:54:56 -08:00
Aseem Bansal
db2f4fb762
chore(ci): speed up ingestion test runs (#7296) 2023-02-10 14:34:35 +05:30
Aseem Bansal
d18dab4084
chore(ci): remove redundant quickstart check (#7286) 2023-02-09 19:47:24 +05:30
Aseem Bansal
2ae368cdbe
chore(ci): ensure kafka setup runs for smoke tests (#7278) 2023-02-08 09:03:59 -06:00
Aseem Bansal
8df66ecaff
fix(ci): trivy code scanning fix (#7232) 2023-02-02 21:50:24 +05:30
Harshal Sheth
5d7246b151
ci(): fix datahub-upgrade quickstart regression (#7217)
Co-authored-by: John Joyce <john@acryl.io>
2023-02-01 15:30:18 -08:00
Harshal Sheth
7e869cd1d6
feat(): add a linter for github actions workflows (#7178)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-30 13:13:41 -08:00
Harshal Sheth
fe910e5bcf
ci(ingest): fix broken CI workflow for metadata-ingestion (#7161) 2023-01-29 17:52:41 +05:30
Aseem Bansal
e880f1415a
fix(ci): do not fail fast for matrix runs (#7132) 2023-01-25 09:31:47 -08:00
Pedro Silva
bef59b0928
feat(quickstart): Remove kafka-setup as a hard deployment requirement (#7073) 2023-01-24 16:12:57 +00:00
Aseem Bansal
db968497cc
fix(analytics): add missing usage events causing warning in logs (#7109) 2023-01-24 17:15:14 +05:30
david-leifker
1452332835
fix(build): Fix postgres setup gha (#7104) 2023-01-23 12:41:39 -06:00
david-leifker
5e935c7ae8
fix(build): postgres docker secret (#7092) 2023-01-20 15:15:47 -06:00
Amanda Ng
087b169b38
feat(editor): improve documentation editor using Remirror (#6631)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2023-01-17 18:00:39 -05:00
Aseem Bansal
10db6265c0
fix(ci): use trivy offline scanning (#7050) 2023-01-16 12:55:32 -06:00
Aseem Bansal
16c41b4778
fix(ci): only scan on master branch (#7047) 2023-01-16 09:34:37 -06:00
Aseem Bansal
e8a074f8b8
chore(ci): parallelise build and test workflow to reduce time (#6949) 2023-01-05 14:45:04 +05:30
david-leifker
c845866521
test(misc): misc test updates (#6890) 2022-12-29 11:26:42 -06:00
Aseem Bansal
0bbe6d693e
fix(ci): reduce flakiness in add_users, siblings smoke test (#6883) 2022-12-29 21:47:25 +05:30
Peter Szalai
9c794af4a7
feat(CI): add venv caching (#6843)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-28 13:22:47 +01:00
Harshal Sheth
8e0f4078b8
chore(ingest): loosen additional requirements (#6868) 2022-12-28 12:08:53 +01:00
Aseem Bansal
1bec1d87bd
fix(ci): reduce smoke test run time (#6841) 2022-12-27 00:08:01 +05:30
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) (#6744)
* fix(security): commons-text in frontend

* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint

* testing docker build

* Add retry configuration options for entity client

* Kafka debugging

* fix(kafka-setup): parallelize topic creation

* Adjust docker build

* Docker build updates

* WIP

* fix(lint): metadata-ingestion lint

* fix(gradle-docker): fix docker frontend dep

* fix(elastic): fix race condition between gms and mae for index creation

* Revert "fix(elastic): fix race condition between gms and mae for index creation"

This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.

* fix(test): fix datahub frontend test for clean/test cycle

* fix(test): datahub-frontend missing assets in test

* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer

* gitingore update

* fix(docker): remove platform on docker base image, set by buildx

* refactor(kafka-producer): update kafka producer tracking/logging

* updates per PR feedback

* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool

Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Aseem Bansal
6fdbf6b42d
Revert "fix(ci): remove warnings due to deprecated action (#6735)" (#6857)
This reverts commit 1da27edcbc97d64d5e88321c8727fabf92a576f8.
2022-12-26 17:56:46 +05:30
Shirshanka Das
b236d0958c
feat(ci): adding github security reporting to trivy scans (#6773) 2022-12-15 21:28:07 -08:00
Aseem Bansal
1da27edcbc
fix(ci): remove warnings due to deprecated action (#6735) 2022-12-13 10:42:21 +05:30
david-leifker
68e30d3bc2
test(quickstart): add debugging to quickstart test (#6718) 2022-12-09 09:26:22 -06:00
Harshal Sheth
1165884c6b
ci: stop commenting unit test results on PR (#6700)
The results will still be surfaced under the "Test Results" action
workflow, but the results won't be commented on the PR itself.
2022-12-08 23:19:27 +01:00
Aseem Bansal
5b52534fd4
fix(ci): reduce warnings due to deprecated action (#6686) 2022-12-08 10:00:05 +05:30
mohdsiddique
b46d52dc0f
feat(gms): Enable auth-api publishing to maven (#6671) 2022-12-06 21:56:36 -08:00
Harshal Sheth
fceef480a2
chore(ingest): remove feast-legacy (#6661) 2022-12-06 14:19:38 -08:00
Aseem Bansal
0862a22632
fix(build): remove warnings in github actions (#6512) 2022-11-23 16:43:23 +05:30
Harshal Sheth
3e907ab0d1
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ (#6204)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-11 15:04:36 -05:00
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x (#6331) 2022-11-01 21:12:34 -07:00
Kevin Neville
3eae087a30
chore(ci): update actions versions addressing warnings in builds. (#6308) 2022-10-28 17:10:30 -07:00
Aseem Bansal
98eba90edc
fix(build): allow image tag via env, fix requirements (#6237) 2022-10-19 12:26:22 +05:30
Shirshanka Das
48b46971b8
feat(ingest): remove hardcoded env variable default for cli version (#6075) 2022-10-02 16:46:34 -07:00
leifker
203a6ff57b
refactor(java11) - convert most modules to java 11 (#5836)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-25 08:39:22 -07:00
Tamas Nemeth
46494bc0ad
feat(ingest): presto-on-hive - Description and hive view support (#5993)
* Presto on Hive Improvements like:
 - Capturing table/column descriptions
 - Adding Hive view support
 - Various bug fixes
2022-09-21 00:40:10 +02:00
Shirshanka Das
e24e5b61b9
fix(datahub-ranger-plugin): add support to publish jars (#5983) 2022-09-19 17:46:01 -07:00
Harshal Sheth
2f7fe593b7
ci: reduce pip backtracking in airflow plugin (#5982) 2022-09-19 12:40:51 -07:00