3652 Commits

Author SHA1 Message Date
John Plaisted
6a067fc25c
docs: Update for topic name configuration (#1909)
* docs: Update for topic name configuration

* Move kafka ssl documentation from faq to kafka config page.
2020-10-02 09:15:07 -07:00
Mars Lan
7e3f443672
Update README.md 2020-10-01 12:17:27 -07:00
Mars Lan
928444928a
Update README.md 2020-09-30 13:00:36 -07:00
Mars Lan
24b0e03b8a
Update developers.md 2020-09-30 06:45:03 -07:00
Mars Lan
a13ca65e02
Update README.md 2020-09-30 06:20:14 -07:00
Mars Lan
864c2d5723
Update links.md 2020-09-30 04:44:29 -07:00
Mars Lan
68b8718488
Update links.md 2020-09-30 04:39:31 -07:00
Shirshanka Das
69b6320005
Update Readme.md
Adding Crunch talk video to intro, fix typo.
2020-09-29 23:26:43 -07:00
Charlie Tran
9c128ae6a3
frontend code migration and unused code removal font update and minor improvements (#1904) 2020-09-29 16:04:25 -07:00
John Plaisted
54eee3efd3
Add new spring factories to customize metadata event topic names. (#1894)
New env vars:

- METADATA_CHANGE_EVENT_NAME: The name of the metadata change event topic.
- METADATA_AUDIT_EVENT_NAME: The name of the metadata audit event topic.
- FAILED_METADATA_CHANGE_EVENT_NAME: The name of the failed metadata change event topic.

This will need to be consistent throughout your ecosystem.

CLOSES: #1840
2020-09-29 13:59:34 -07:00
Mars Lan
b7f846d13b
docs: update links (#1903) 2020-09-29 07:30:35 -07:00
Mars Lan
c67beb8b91
docs: add Budapest talk (#1901) 2020-09-29 06:07:58 -07:00
Mars Lan
33dd93f66f
Add files via upload 2020-09-29 06:04:48 -07:00
Mars Lan
c413a2a69c
build: fix build by adding zookeeper dependency explicitly (#1900) 2020-09-29 06:00:47 -07:00
Lars Nielsen
b26d6fe880
Bump up kafkaAvroSerde to support SSL for Schema Registry (#1898)
* Bump up kafkaAvroSerde to support security config for Confluent Schema Registry

* Support certs in secrets

* Extra Spring config (e.g. security)

* Optional values

* Clarify log warnings

* Update faq.md

Co-authored-by: Lars Nielsen <Lars.Nielsen@kindredgroup.com>
Co-authored-by: Mars Lan <mars.th.lan@gmail.com>
2020-09-29 04:12:43 -07:00
Grant Nicholas
9bcf273661
fix(docker): update mae and mce consumer images to include glibc compat layer. allows the consumer jobs to deal with snappy compressed kafka topics when running on alpine linux (#1899) 2020-09-28 15:30:56 -07:00
John Plaisted
2f86cd680e
[BREAKING] Break dependency of ebean-dao on metadata-models. (#1895)
The coupling was between the static path extractor API. This broken by making a new `UrnPathExtractor` interface, and adding an overload of `EbeanLocalDAO`'s constructor to accept one (no breaking constructor change). The old constructors default to an `EmptyPathExtractor`, which does nothing (which is a breaking behavioral change, see below).

BREAKING: `DatasetUrnPathExtractor` was deleted. No one should've been depending on this directly. However, downstreams that were relying on it being there at runtime (dataset GMS) need to copy `DatasetUrnPathExtractor` and create their `EbeanLocalDAO` with one. Note that this is a little dangerous becasue it is a runtime behavioral change only. Potential impact is that SCSI suddenly stops working as intended.

SYNC=metadata-models_101.0.0
2020-09-28 12:30:34 -07:00
Mars Lan
b6d26f7fac
Update links.md 2020-09-28 12:09:43 -07:00
Mars Lan
a15384ab41
Update README.md 2020-09-28 12:08:55 -07:00
Mars Lan
2d9729912e
Update links.md 2020-09-28 04:55:39 -07:00
Mars Lan
fcfc5b71fb
Add files via upload 2020-09-28 04:49:47 -07:00
Mars Lan
5dfd38fb72
Update townhall-history.md 2020-09-27 09:31:17 -07:00
Mars Lan
4410a95455
docs: update town hall history (#1897) 2020-09-27 09:29:40 -07:00
Mars Lan
12f0c5b1ad
Update README.md 2020-09-27 07:24:35 -07:00
Liangjun Jiang
6bb5d24c35
add default KAFKA_BOOTSTRAP_SERVER (#1893)
Co-authored-by: Liangjun <liajiang@expediagroup.com>
2020-09-25 19:55:48 -07:00
John Plaisted
821bce7d69
feat: Port mce-cli to Java. (#1871)
Port mce-cli to Java.

Also moved off the avro format event file to json instead. Much nicer to use :)
2020-09-25 14:05:29 -07:00
John Plaisted
48b47a7a59
Update for October 2020-09-25 12:12:21 -07:00
John Plaisted
ab5d409206
Add September 2020-09-25 12:10:52 -07:00
Jyoti Wadhwani
dc9c877984 [scsi] preserve the order of urns 2020-09-24 16:02:12 -07:00
John Plaisted
5e70f3648c Fix build after sync.
- Add build files for new module.
- Correctly edit TestUtils since it isn't synced.
- Reference new test utils.
- Delete duplicate pdl files.

SYNC=metadata-models_100.0.1
2020-09-24 16:02:12 -07:00
John Plaisted
542ae67cb1 Add support for customizing topic names via a convention.
Requested by a few people in OS. See https://github.com/linkedin/datahub/issues/1840.

Companies need full customization over the topic name. This new class should be easily customizable by using a spring factory.

TODO to finish the implmentation for v5. For right now v5 is LI only and unfinished. Getting this in for v4 so it is useful to other companies now.

TODO AFTER OPEN SOURCE PUSH - make configurable via spring
TODO AFTER SUBMIT - see where else we can use this (jobs, where else?)
2020-09-24 16:02:12 -07:00
Jyoti Wadhwani
b944910b0e extend filter finder method to get metadata from SCSI 2020-09-24 16:02:12 -07:00
Jyoti Wadhwani
c3c52cf8f6 add support for getWithExtraInfo in BaseLocalDAO 2020-09-24 16:02:12 -07:00
John Plaisted
8d536a54d3 Break dependency of metadata-dao on metadata-models.
This also breaks a few others transitively.
2020-09-24 16:02:12 -07:00
John Plaisted
e4ce0376d2 Fix open source build.
Do not use internal URN API.
2020-09-24 16:02:12 -07:00
John Plaisted
96da83033c Break dependency of metadata-test-utils on metadata-models. 2020-09-24 16:02:12 -07:00
Jyoti Wadhwani
f07420a434 change finder return type for listing urns from SCSI 2020-09-24 16:02:12 -07:00
Jyoti Wadhwani
9d402574e2 use finder for list urns from index 2020-09-24 16:02:12 -07:00
Jyoti Wadhwani
caf22dc921 add support for conditions other than equals 2020-09-24 16:02:12 -07:00
Kerem Sahin
e042289b81
Update graph-onboarding.md 2020-09-23 14:58:35 -07:00
John Plaisted
1881b06838
Update agenda with times and people links 2020-09-23 10:05:56 -07:00
Fredrik Sannholm
d50b9c01b4
fix (docker): Fix install of Chrome in frontend Dockerimage (#1889)
* fix (docker): Fix install of Chrome in frontend Dockerimage

Retry installing Chrome after dependencies have been installed

* fix (docker): Install Chrome with apt-get

Install Chrome and dependencies at the same time, using apt-get
2020-09-22 12:02:37 -07:00
Kerem Sahin
27baa10004
Update townhall-history.md 2020-09-21 11:42:55 -07:00
John Plaisted
d6c2b5f56e
Update with September Agenda 2020-09-21 11:24:06 -07:00
Mars Lan
a897f7a3bf
build: add failure notification on push (#1873)
* build: add failure notification on push

* build: fix build (#1872)

* build: fix build

`kafkaSerializers` is already included transitively via `kafkaAvroSerde`

* Update build.gradle
2020-09-21 10:29:24 -07:00
Charlie Tran
57fdc5c00c
Adds ability for midtier to serve custom dataset properties from aspect (#1881) 2020-09-20 11:04:51 -07:00
Mars Lan
205a6c920b
Update debugging.md 2020-09-18 04:21:56 -07:00
Charlie Tran
3b7eb08284
Fixes current user entity not being populated correctly (#1880) 2020-09-17 23:03:15 -07:00
Jyoti Wadhwani
4f809fb11b
more changes on faceted search 2020-09-17 16:51:17 -07:00
Jyoti Wadhwani
c9deda8075
update the doc for facet field 2020-09-17 16:22:38 -07:00