david-leifker
9b14c1800b
chore(security): removing unmaintained es7 upgrade proces ( #7790 )
2023-04-13 10:00:58 -05:00
david-leifker
d7faaa00c0
fix(contrib): Update base python image for es7-upgrade ( #6562 )
2022-11-28 15:51:03 -06:00
Shirshanka Das
238fca5191
chore: remove unnecessary modules from codebase ( #5420 )
2022-07-17 22:02:45 -07:00
Remi
79482262ed
feat(ingest): add support for Looker view built from SQL-based derived tables ( #2478 )
2021-05-03 07:54:57 -07:00
Dexter Lee
ae4def2f25
feat(k8s): Move helm charts out of contrib ( #2440 )
2021-04-23 00:31:06 -07:00
Dexter Lee
c46d842981
fix(k8s): Add credentials to kafka-setup job and clean up ( #2415 )
2021-04-18 11:07:19 -07:00
Dexter Lee
379c2e1c0c
feat(k8s): add extraEnvs to setup jobs ( #2404 )
2021-04-16 09:39:40 -07:00
Dexter Lee
3ddf163a91
feat(k8s): Add ability to add service accounts to setup jobs ( #2390 )
2021-04-13 07:59:18 -07:00
Dexter Lee
f293add5b1
feat(k8s): Update pods with correct probes and remove unnecessary dependencies ( #2362 )
2021-04-09 17:49:12 -07:00
Fredrik Sannholm
f837360b9a
chore(dependabot): Update pyyaml version ( #2365 )
2021-04-08 11:43:19 -07:00
Grant Nicholas
fce8a535c2
feat(ingest): adds experimental support for ingesting Looker metadata ( #2315 )
2021-04-02 11:00:58 -07:00
Harshal Sheth
7085051a73
fix(ingest): remove outdated metadata-ingestion scripts ( #2325 )
2021-03-31 20:50:21 -07:00
shakti-garg
1a62813e71
fix(k8s): make es-setup job parameters more contextual ( #2313 )
...
* 2301 | update property names to signify that it is a setup job (#2313 )
2021-03-31 12:32:03 -07:00
bernardino
9a947948f0
feat(Kubernetes): Add JMX exporter containers to all DataHub components ( #2251 )
...
Co-authored-by: Ricardo Bernardino <ricardo.bernardino@definedcrowd.com>
2021-03-30 22:03:27 -07:00
shakti-garg
918097c874
feat(k8s): Add k8s hook in datahub helm chart for setting up elasticsearch ( #2302 )
2021-03-29 12:41:30 -07:00
John Plaisted
5e91014e00
feat(search) BREAKING Support ElasticSearch 7, drop ES5 ( #2263 )
...
Merges in changes from our ES7 branch, and drops support for ES5.
This is a breaking change due to the upgrade, we have a ES5 branch at the commit before this.
2021-03-18 19:16:44 -07:00
Pedro Silva
45d622b945
feat(k8s): Add metadata-ingestion as a Helm component ( #2236 )
...
Co-authored-by: Pedro Silva <pedro.silva@definedcrowd.com>
2021-03-15 22:32:28 -07:00
John Plaisted
2894e2b348
(feat) Simple python script to carry over ES indices from 5 to 7. ( #2192 )
2021-03-08 15:16:14 -08:00
Harshal Sheth
20bf794ec4
docs: hosted documentation website ( #2174 )
2021-03-05 00:12:12 -08:00
Javier Sotelo
77d86f7f79
Support optional initContainers to setup extra initialization ( #2058 )
...
Co-authored-by: jsotelo <javier.sotelo@viasat.com>
2021-01-13 19:59:49 -08:00
shakti-garg-saxo
808330f7f8
feat(kubernetes): Add pod-level annotations to the datahub helm charts ( #2005 )
...
* 2003 | add pod annotations
* 2003 | update documentation and bump up chart version
Co-authored-by: shakti-garg <shakti.garg@gmail.com>
2020-11-17 15:20:55 -08:00
shakti-garg-saxo
1351de4599
1995 | fix indentation value in helm deployment templates ( #2004 )
...
Co-authored-by: shakti-garg <shakti.garg@gmail.com>
2020-11-17 07:29:59 -08:00
Lars Nielsen
b26d6fe880
Bump up kafkaAvroSerde to support SSL for Schema Registry ( #1898 )
...
* Bump up kafkaAvroSerde to support security config for Confluent Schema Registry
* Support certs in secrets
* Extra Spring config (e.g. security)
* Optional values
* Clarify log warnings
* Update faq.md
Co-authored-by: Lars Nielsen <Lars.Nielsen@kindredgroup.com>
Co-authored-by: Mars Lan <mars.th.lan@gmail.com>
2020-09-29 04:12:43 -07:00
John Plaisted
821bce7d69
feat: Port mce-cli to Java. ( #1871 )
...
Port mce-cli to Java.
Also moved off the avro format event file to json instead. Much nicer to use :)
2020-09-25 14:05:29 -07:00
John Plaisted
6ece2d6469
Start adding java ETL examples, starting with kafka etl. ( #1805 )
...
Start adding java ETL examples, starting with kafka etl.
We've had a few requests to start providing Java examples rather than Python due to type safety.
I've also started to add these to metadata-ingestion-examples to make it clearer these are *examples*. They can be used directly or as a basis for other things.
As we port to Java we'll move examples to contrib.
2020-09-11 13:04:21 -07:00
shakti-garg-saxo
236d5e6271
feat(kubernetes): Improve the security of the kubernetes/helm charts ( #1782 )
...
* 1747 | remove obsolete yaml files
* 1747 | remove configmap and its hardcoded references
* 1747 | add missing input parameter of neo4j.host
* 1747 | remove obsolete secrets and parameterize the rest
* 1747 | auto-generate gms secret
* 1747 | remove fullName overrides
* 1747 | fix parameters in subchart's values.yaml
* 1747 | remove hardcoding from parameters for gms host and port
* 1747 | upgrade chart version
* 1747 | update helm docs
* 1747 | add extraEnv, extraVolume and extraMounts
* 1747 | Alters pull policy of images to 'always' for ldh
Co-authored-by: shakti-garg <shakti.garg@gmail.com>
2020-08-20 02:42:22 -07:00
Chris Lee
4143fb901e
<refactor>[ingestions]: align the default kafka topics with PR #1756 ( #1758 )
2020-07-29 20:26:01 -07:00
Javier Sotelo
aa65b46be7
fix: add missing neo4j.host helm var ( #1698 )
...
Co-authored-by: jsotelo <javier.sotelo@viasat.com>
2020-06-11 12:03:51 -07:00
Javier Sotelo
e7be6db936
fix: remove helm container command ( #1697 )
...
The current helm command is not compatible with the
docker image. Instead, rely on the docker image's CMD.
Co-authored-by: jsotelo <javier.sotelo@viasat.com>
2020-06-11 10:48:53 -07:00
Mars Lan
920a1774dc
docs: points to docker images hosted by linkedin org ( #1683 )
...
As we're now utilize GitHub Actions to build & publish docker images to docker hub under linkedin org
Also allow overriding image tags via DATAHUB_VERSION environment variable
2020-06-01 09:36:51 -07:00
Liangjun Jiang
ec5fbbc496
Add openldap-etl script and instruction ( #1647 )
...
* add openldap-etl scripts, README.md and docker compose file
* remove a misleading comment
* fixed a typo in README.md
Co-authored-by: Liangjun <liajiang@expediagroup.com>
2020-04-24 14:28:39 -07:00
larluo
c7851b7df4
migrate hive generator ( #1641 )
...
* migrate hive generator
2020-04-21 20:29:19 -07:00
larluo
c97d3fc102
add postgresql support and force utf8 encode on non-utf8 locale ( #1637 )
2020-04-18 11:19:54 -07:00
larluo
ee791b7737
add field for ui and parser reference ( #1633 )
2020-04-06 07:29:28 -07:00
Bharat
e7feaf1fc0
fix: Reduce mae-consumer & mce-consumer docker image sizes ( #1632 )
2020-04-05 09:30:05 -07:00
Bharat
51c0a5c6fe
fix: Reduce gms & frontend docker image sizes ( #1630 )
...
* fix: Reduce gms docker image size to 254MB from 1.15GB
* fix: Reduce frontend docker image size to 294MB from 2.68GB
2020-04-04 07:51:29 -07:00
larluo
ad91a528ce
add oracle driver ( #1613 )
2020-03-27 18:28:55 -07:00
larluo
c2e4761753
init jdbc generator ( #1611 )
2020-03-26 21:32:26 -07:00
Bharat
502db03b0d
add support for kubernetes helm packaging ( #1609 )
2020-03-26 19:28:30 -07:00
larluo
3fae73691e
add lineage hive ( #1608 )
2020-03-25 21:54:22 -07:00
Bharat
bfc6c6fd0b
feat: support for Kubernetes-based deployment ( #1602 )
...
* Support for Kubernetes based deployments
* Update README.md
* Updated the README to change the configmap with relevant docker host details
* Fixed README based on the comments in PR
* Fixed README to point to Confluent Kafka Helm charts
Co-authored-by: Mars Lan <mars.th.lan@gmail.com>
2020-03-24 06:01:51 -07:00
larluo
040ed85f54
add datahub producer ( #1597 )
2020-03-18 07:29:47 -07:00
larluo
96e9c4f031
add datahub-check for port check ( #1595 )
2020-03-13 07:51:24 -07:00
larluo
d30b0eba5c
switch home-manager from master to stable version ( #1594 )
2020-03-12 23:18:11 -07:00
larluo
6151949f3d
add nix sandbox ( #1585 )
2020-03-09 20:23:47 -07:00
Mars Lan
2ded8d8434
Update README.md
2020-02-10 05:03:52 -08:00
Mars Lan
531c8412cd
Add contrib directory for community contirbutions
2020-01-01 15:17:26 -08:00
