Aseem Bansal
5755d2ca9e
fix(ingest): okta undefined variable error ( #6882 )
2022-12-29 20:24:22 +05:30
John Joyce
218f3c3414
refactor(docs): Correctly spell elasticsearch in docs ( #6880 )
2022-12-29 15:21:24 +01:00
Aseem Bansal
b8664d6630
fix(lint): pin pydantic version ( #6886 )
2022-12-29 19:36:14 +05:30
Aseem Bansal
1d0bb1cd8a
refactor(smoke): use env variables ( #6866 )
2022-12-29 17:16:43 +05:30
david-leifker
122c024bab
refactor(gradle): move the local docker registry to common location ( #6881 )
2022-12-29 15:08:36 +05:30
Harshal Sheth
667ca8632d
feat(ingest): avoid embedding serialized json in metadata files ( #6742 )
2022-12-28 19:28:38 -05:00
david-leifker
b5275acab8
test(mce-consumer): mockbeans ( #6878 )
2022-12-28 16:18:10 -06:00
John Joyce
1503ef38f9
docs(adding users): Refreshing the docs for adding new DataHub Users ( #6879 )
2022-12-28 13:54:42 -08:00
Meenakshi Kamalaseshan Radha
d9b71cea19
feat(ui) - Add a custom error message for bulk edit to add clarity ( #6775 )
...
Co-authored-by: Kamalaseshan Radha <mkamalas@LAMU02DN212MD6R.uhc.com>
2022-12-28 13:22:21 -08:00
Aseem Bansal
1fb46b82c8
chore(ci): update dependencies docs-website ( #6871 )
2022-12-28 13:21:47 -08:00
Harshal Sheth
b474315e07
fix(ingest): conditionally include env in assertion guid ( #6811 )
2022-12-28 11:35:20 -08:00
david-leifker
d8af0b43bd
fix(runid): always update runid, except when queued ( #6876 )
2022-12-28 13:09:30 -06:00
Mayuri Nehate
2129496c98
feat(ingest/snowflake): handle failures gracefully and raise permission failures ( #6748 )
2022-12-28 08:20:37 -08:00
Peter Szalai
9c794af4a7
feat(CI): add venv caching ( #6843 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-28 13:22:47 +01:00
jx2lee
988b3d476c
docs(google-analytics): Correct grammatical error in README.md ( #6870 )
2022-12-28 12:01:53 +00:00
Tamas Nemeth
25b5a12b9d
feat(ingest): bigquery/snowflake - Store last profile date in state ( #6832 )
2022-12-28 12:09:18 +01:00
Harshal Sheth
8e0f4078b8
chore(ingest): loosen additional requirements ( #6868 )
2022-12-28 12:08:53 +01:00
david-leifker
cb6314c51d
fix(elasticsearch): fixes out of order runId writes ( #6845 )
...
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-27 17:50:11 -06:00
cccs-eric
ec8a4e0eab
feat(ingest): upgrade pydantic version ( #6858 )
...
This PR also removes the requirement on docker-compose v1 and makes our tests use v2 instead.
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-27 17:06:16 -05:00
Harshal Sheth
d851140048
chore(ingest): loosen requirements file ( #6867 )
2022-12-27 22:39:12 +01:00
Mayuri Nehate
14b48489d4
feat(ingest): pass timeout config in kafka admin client api calls ( #6863 )
2022-12-27 12:45:11 -08:00
david-leifker
a5b5abf270
fix(kafka): datahub-upgrade job ( #6864 )
2022-12-27 12:35:48 -06:00
Aseem Bansal
bef87abd6c
refactor(graphql): make graphqlengine easier to use ( #6865 )
2022-12-27 22:58:12 +05:30
Harshal Sheth
31260888fc
feat(ingest/airflow): support raw dataset urns in airflow lineage ( #6854 )
...
* feat(ingest/airflow): support dataset Urns in airflow lineage
This PR also
- resolves a reported circular import issue
- refactors the Airflow tests to reduce duplication
* fix test
2022-12-27 08:59:26 +01:00
Mayuri Nehate
69a2347db1
feat(ingest): update profiling to fetch configurable number of sample values ( #6859 )
2022-12-27 08:57:26 +01:00
david-leifker
10ea10ce85
fix(security): require signed/encrypted jwt tokens ( #6565 )
...
* fix(security): require unsigned/encrypted jwt tokens
* Add import
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 19:45:32 +00:00
Aseem Bansal
1bec1d87bd
fix(ci): reduce smoke test run time ( #6841 )
2022-12-27 00:08:01 +05:30
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Aseem Bansal
6fdbf6b42d
Revert "fix(ci): remove warnings due to deprecated action ( #6735 )" ( #6857 )
...
This reverts commit 1da27edcbc97d64d5e88321c8727fabf92a576f8.
2022-12-26 17:56:46 +05:30
John Joyce
cc0b6019fb
chore(0.9.5): Bump defaults for release v0.9.5 ( #6856 )
2022-12-26 15:40:26 +05:30
Harshal Sheth
392115b4c4
feat(ingest): add pydantic helper for removed fields ( #6853 )
2022-12-26 15:31:49 +05:30
John Joyce
c482ef0215
refactor(ui): Misc domains improvements ( #6850 )
2022-12-23 11:06:47 -08:00
Harshal Sheth
ea5ee6f761
fix(ingest/looker): handle missing label fields ( #6849 )
2022-12-22 19:43:44 -05:00
mohdsiddique
9daa8ed56f
feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion ( #6829 )
...
* lint fix
* domain in term
* domain in term
* review comments
* add todo
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-22 17:47:57 -05:00
RyanHolstien
53fe13ebed
docs(confluent): add details for actions pod for confluent ( #6810 )
2022-12-22 14:40:26 -08:00
david-leifker
bacc2f957b
fix(oidc): fix oidc authentication loop ( #6848 )
...
* fix(oidc): fix oidc authentication loop
2022-12-22 16:12:51 -06:00
Harshal Sheth
1d0c7852a7
feat(ingest): add db/schema properties hook to SQL common ( #6847 )
2022-12-22 13:38:59 -08:00
Chris Collins
2ef2ad05d0
fix(ingestion) Inject pipeline_name into recipes at runtime ( #6833 )
2022-12-22 13:16:51 -08:00
John Joyce
4cba09e97d
fix(ingest): Fixing lint ( #6844 )
2022-12-22 08:33:18 -08:00
wangsaisai
0f8e2d945e
fix(ingest): kafka ingest task hand up with error bootstrap server ( #6820 )
2022-12-22 07:39:30 -08:00
Aseem Bansal
bdd346db27
fix(ci): add labels based on more folders ( #6840 )
2022-12-22 07:33:02 -08:00
Teppo Naakka
b0a82ca7f4
feat(ui): sortable domain list ( #6736 )
2022-12-22 07:22:22 -08:00
Aseem Bansal
86ee0e58de
feat(ci): add cypress test ui based ingestion ( #6769 )
2022-12-22 20:37:29 +05:30
John Joyce
8976844635
feat(ui): Support adding custom id when creating term and term group ( #6830 )
2022-12-22 06:32:46 -08:00
danielli-ziprecruiter
a6470fc267
feat(ingestion): spark - support lineage for delta lake writes ( #6834 )
2022-12-22 12:13:32 +01:00
Mirko R
3b8686d012
docs: Incorrect import statement fixed in example ( #6838 )
2022-12-22 12:08:20 +01:00
Mayuri Nehate
a05c5c4069
feat(ingest): extract kafka topic config properties as customProperties ( #6783 )
2022-12-22 09:34:55 +01:00
david-leifker
2a182f4846
fix(pac4j-oidc): add verifier parameter ( #6835 )
...
* fix(pac4j-oidc): add verifier parameter
2022-12-21 20:11:11 -06:00
Chris Collins
517688d050
fix(lineage) Fix lineage viz with multiple siblings ( #6826 )
2022-12-21 14:36:09 -05:00
John Joyce
2e3a25123d
refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker ( #6002 )
2022-12-21 08:02:59 -08:00
