60 Commits

Author SHA1 Message Date
Esteban Gutierrez
8a17ba14d6
fix(): Fixes multiple minor security vulnerabilities (#13222)
bug(snappy): Make sure right snappy version is installed
fix(docker): update Dockerize to version v0.9.3
fix(gms): fixes dgraph4j netty deps
fix(docker): remove SGID on /home/datahub and /home/datahub-integration
fix(datahub-actions): bump setuptools and wheel version
fix(docker): update c-ares version
fix(docker): datahub-actions addendum
2025-04-21 16:39:26 -05:00
Andrew Sikowitz
d138a64a6a
ci(graphql,workflows): Format .md, .graphql, and workflow .yml files via prettier (#13220) 2025-04-16 16:55:51 -07:00
Chakru
cf40116680
CI speedup (#13057) 2025-04-07 10:13:07 +05:30
Chakru
40106be208
build: optimizations for incremental builds and faster CI (#13033)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2025-04-02 11:51:10 +05:30
david-leifker
210e2c1fa2
chore(alpine): bump alpine images 3.21 (#12302) 2025-01-09 14:32:18 -06:00
david-leifker
327c6f911a
chore(bump): bump alpine and dockerize (#12184) 2024-12-20 15:59:07 -06:00
Aseem Bansal
082ef5b351
build: fix docker warnings (#11163) 2024-08-14 14:23:11 +05:30
david-leifker
033114d522
chore(alpine): update alpine base image (#10754) 2024-06-20 11:12:36 -05:00
david-leifker
bcb40d3d71
chore(security): updates for security vulnerabilities (#10740) 2024-06-18 11:28:39 -05:00
Davi Arnaut
927775d2fa
feat(opensearch): support for updating ISM policy used for usage events (#10018) 2024-03-08 10:28:45 -08:00
david-leifker
943bb57cbc
feat(backend): structured properties and forms (#9626)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2024-01-22 11:46:04 -06:00
noggi
3e79a1325c
Pin alpine base image version to 3.18 (#9421) 2023-12-08 15:37:12 -06:00
david-leifker
c66619ccc7
fix(elasticsearch): set datahub usage events shard & replica count (#9388) 2023-12-05 14:03:24 -06:00
Hendrik Richert
966cb175f7
feat(dev): Make repositories configurable for enterprise developers (#9230)
Co-authored-by: Hendrik Richert <hendrik.richert@swisscom.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-11-28 14:52:11 -06:00
david-leifker
ff90fb633d
fix(datahub-ingestion): prevent transitive deps, bump addtional pyspa… (#9233) 2023-11-13 16:26:53 -06:00
david-leifker
efd73a5f57
fix(elasticsearch): fix elasticsearch-setup for dropped 000001 index (#9074) 2023-11-03 16:19:39 -05:00
RyanHolstien
6f466b0e50
chore(docker): update base images to alpine 3.18 (#8967) 2023-10-17 15:40:38 -05:00
david-leifker
40caf35812
feat(datahub-ingestion): refactor datahub ingestion slim images (#8515) 2023-08-17 10:40:17 +05:30
Kevin Meyer
34f8f17a55
fix(PFP-260): Upgrading sqlite to fix SQLITE-449762 (#8032) 2023-05-12 13:03:43 -04:00
david-leifker
571970cf62
Dockerize updates (#7387) 2023-02-22 15:43:11 +05:30
david-leifker
267472ccbb
gradle(development): add additional commands for development (#7321) 2023-02-16 10:41:42 -06:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements (#6894) 2023-01-31 18:44:37 -06:00
tony csoka
1149cea1c8
fix(docker): support non amd64 dockerize in setup containers (#7091)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-23 10:25:33 +00:00
Peter Szalai
a164bdabe2
fix(es-setup): create data stream on non-aws (#6926) 2023-01-03 09:52:45 +01:00
david-leifker
122c024bab
refactor(gradle): move the local docker registry to common location (#6881) 2022-12-29 15:08:36 +05:30
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) (#6744)
* fix(security): commons-text in frontend

* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint

* testing docker build

* Add retry configuration options for entity client

* Kafka debugging

* fix(kafka-setup): parallelize topic creation

* Adjust docker build

* Docker build updates

* WIP

* fix(lint): metadata-ingestion lint

* fix(gradle-docker): fix docker frontend dep

* fix(elastic): fix race condition between gms and mae for index creation

* Revert "fix(elastic): fix race condition between gms and mae for index creation"

This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.

* fix(test): fix datahub frontend test for clean/test cycle

* fix(test): datahub-frontend missing assets in test

* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer

* gitingore update

* fix(docker): remove platform on docker base image, set by buildx

* refactor(kafka-producer): update kafka producer tracking/logging

* updates per PR feedback

* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool

Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Pedro Silva
c93bdc5e39
fix(setup): Bump setup images to alpine 3.14 with arch based on machine OS. (#6612)
* fix(setup): Bump setup images to alpine 3.14 with arch based on machine OS.
2022-12-09 09:49:24 -06:00
djordje-mijatovic
9e6489f603
refactor(elastic-setup): Elasticsearch setup analytics (#6291) 2022-10-31 13:50:54 -07:00
Tomáš Kubín
596d48448e
fix(elasticsearch) Analytics indices creation on AWS ES (#5502)
* refactor(elasticsearch-setup-job): create-indices.sh readability

The script contains many copy-pasting and is not easy to follow.
Add comments, extract commonly used operations into functions, unify approaches.

* fix(elasticsearch-setup-job): AWS indices creation

Fix the issue where Amazon OpenSearch (AWS ES) indices are incorrectly initialised
and the Analytics screen shows errors only.

* feat(elasticsearch-setup-job): configuration hint

mention USE_AWS_ELASTICSEARCH env value if it seems it's set the wrong way

* fix(elasticsearch-setup-job): silent curl

* fix(elasticsearch-setup-job): better USE_AWS_ELASTICSEARCH hint

* docs(elasticsearch-setup-job): index dropping explained

- more comments
- more defensive approach
- index file renamed

* fix(elasticsearch-setup-job): script fixes

* merge(elasticsearch-setup-job): merging in PR #5937

* merge(elasticsearch-setup-job): merging in PR #5963

* merge(elasticsearch-setup-job): merging in PR #5975

Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-09-29 14:10:51 +01:00
Pedro Silva
f83eb6f759
fix(elastic-setup) Fixing env var logic (#6079) 2022-09-29 10:53:45 +05:30
GYUHOON KIM
b45d5eb56a
fix(elasticsearch_index): create datahub_usage_event index where datahub_analytics_enabled set to false (#5974) 2022-09-27 10:20:31 -07:00
Shirshanka Das
6f4835688e
fix(quickstart): elasticsearch-setup script fails on curl (#5975) 2022-09-18 17:05:13 -07:00
Bogdan Antoniu
a86c966c91
feat(elasticsearch-setup): Add insecure option for curl
Co-authored-by: EXTERNAL Antoniu Bogdan (Fortech, EB-DB/ENG-Ku) <external.Bogdan.Antoniu@ro.bosch.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2022-09-16 11:22:50 -05:00
Aseem Bansal
a4720f8e2e
feat(elastic-setup): add better error handling (#5963) 2022-09-16 17:34:06 +05:30
Aseem Bansal
f669ba6f14
feat(elastic-setup): more verbose logging (#5937) 2022-09-14 17:35:26 +05:30
Navin Sharma
ee7c5f75a2
Add Changes to support smoke test on Datahub deployed on kubernetes Cluster (#5334)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2022-07-14 22:04:06 +05:30
Dexter Lee
d4d621800b
fix(docker): fix issues with base64 not working on some platforms (#3723) 2021-12-13 09:39:48 -08:00
Dexter Lee
0576b76eb3
fix(elasticsearch): Use auth tokens to authorize curl requests in dockerize (#3596) 2021-12-07 17:50:08 -08:00
John Joyce
f49666a230
feat(auth): Metadata Service Authentication! (#3598) 2021-11-22 16:33:14 -08:00
Claudio Benfatto
2c63efa7ac
fix(docker): use wait-http-header to avoid printing cleartext credentials (#3241) 2021-09-15 18:11:51 -07:00
John Joyce
2c5edd88ab
feat(graphql): migrating GraphQL API to metadata-service (nee GMS) (#3131) 2021-08-20 10:58:07 -07:00
Dexter Lee
65d6c474f6
fix(docker): upgrade base image version (#3072) 2021-08-10 22:22:36 -07:00
Dexter Lee
acb638cb65
fix(analytics): Fix SSL issue with analytics on frontend (#2840) 2021-07-08 15:11:40 -07:00
Dexter Lee
b4cf74bcb4
fix(analytics): add support for AWS ES (#2539) 2021-05-12 10:54:18 -07:00
John Joyce
29832e5385
feat(Product Analytics): Introducing In-App Analytics Beta (#2499)
Co-authored-by: Harshal Sheth <harshal@acryl.io>
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-05-11 15:41:42 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields (#2228)
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
2021-05-10 13:20:23 -07:00
Dexter Lee
1fc532d831
feat(index): Add index naming convention for elasticsearch (#2386) 2021-04-13 07:56:31 -07:00
shakti-garg
3fb71acf71
feat(es-setup): add logic in elasticsearch setup to compare-and-update index if already exists (#2312)
* 2310 | add logic in es-setup script to compare-and-update index if already exists
2021-04-03 11:17:16 -07:00
Fredrik Sannholm
4591667e05
feat(elasticsearch): Elasticsearch simple auth over https support (#2303) 2021-03-27 20:21:10 -07:00
John Plaisted
5e91014e00
feat(search) BREAKING Support ElasticSearch 7, drop ES5 (#2263)
Merges in changes from our ES7 branch, and drops support for ES5.

This is a breaking change due to the upgrade, we have a ES5 branch at the commit before this.
2021-03-18 19:16:44 -07:00