yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-08-19 06:38:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,798 Commits 243 Branches 123 Tags
Commit Graph

11798 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Gabe Lyons
1193e41955
docs(approvals): update approval permission docs (#6143) 
* Update approval-workflows.md

* Update approval-workflows.md
 2022-10-06 19:33:09 -07:00
Aditya Radhakrishnan
e8259788a3
feat(tracking): add telemetry for frontend events (#6129) 2022-10-06 18:56:32 -07:00
Chris Collins
1f1398605c
feat(lineage) Add Column-Level to Lineage Visualization (#6138) 2022-10-06 19:41:18 -04:00
Gabe Lyons
5f0216cfda
temporarily disable fetching input fields for search results (#6139) 2022-10-06 16:23:24 -07:00
Pablo Takara
485d56982a
fix(ingestion): add fallback to trino (#6044) 
Co-authored-by: Pedro Silva <pedro@acryl.io>
 2022-10-06 14:14:45 -07:00
Harshal Sheth
f446ccef2e
fix(ingest): snowflake - use approx count distinct for columns, fix quoting (#6107) 2022-10-06 13:56:29 -07:00
david-leifker
3dc25c14e0
test(KafkaEmitter): Enable ability to run test locally (#6123) 2022-10-06 13:34:11 -07:00
Harshal Sheth
3f84a2050a
fix(ingest): trino - add trino package max version restriction (#6137) 2022-10-06 13:30:28 -07:00
Ankit keshari
11092c73cf
feat(groups & owners) Add ability to edit group name + assign creator as owner of metadata (#6047) 
Adds ability to edit user group names from the UI. Also sets the creator or Glossary Terms, Glossary Nodes, and Domains to be the owner of said item.
 2022-10-06 15:36:39 -04:00
Shirshanka Das
3106e42e89
fix(ingest): file - add configurability for counting all elements before starting ingestion (#6136) 2022-10-06 10:24:23 -07:00
Tamas Nemeth
6db0925b26
fix(ingest): bigquery-beta - fix for missing key error if dataset was empty (#6133) 2022-10-06 09:20:49 -07:00
david-leifker
928f29438a
build: Update JNA for M1 Mac (#6116) 2022-10-05 14:53:38 -07:00
Paul Logan
cc236f5ca2
docs: sending "getting started" direct to quickstart (#6125) 2022-10-05 14:52:04 -07:00
Shirshanka Das
5fa8818dd3
docs(tableau): adding an ingestion video (#6124) 2022-10-05 14:50:44 -07:00
Gabe Lyons
864e64de8f
feat(lineage): show fully qualified task name in lineage UI (#6126) 
* showing fully qualified task name in lineage UI

* slight refactor

* use display name

* fix test
 2022-10-05 13:54:26 -07:00
PrashantKhadke
395d7ea2f5
docs(okta): Added information about AUTH_OIDC_EXTRACT_GROUPS_ENABLED (#6120) 
* Added information about AUTH_OIDC_EXTRACT_GROUPS_ENABLED

* Update configure-oidc-react-okta.md

Co-authored-by: Aditya Radhakrishnan <aditya.radhakrish@gmail.com>
 2022-10-04 21:13:14 -07:00
Harshal Sheth
164e47a2e9
refactor(ingest): remove typing workarounds (#6108) 
Possible now that we're on mypy 0.980.
 2022-10-04 21:12:05 -07:00
Mayuri Nehate
4ee3ef14ef
fix(ingest): ignore usage and operation for snowflake datasets without schema (#6112) 2022-10-04 21:01:11 -07:00
Tamas Nemeth
2f79b50c24
fix(ingest): presto-on-hive - not failing on Hive type parsing error (#6118) 
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
 2022-10-04 20:54:38 -07:00
Harshal Sheth
95afc1d73f
refactor(ingest): use pydantic utilities for NamingPattern (#6013) 
* refactor(ingest): use pydantic utilities for NamingPattern

* clean up replacement logic

* flatten config hierarchy
 2022-10-04 20:53:18 -07:00
Ravindra Lanka
055e4082da
fix(ingestion): fix percent change computation in stale_entity_removal (#6121) 2022-10-04 20:40:59 -07:00
Harshal Sheth
feb9d59b4d
fix(ingest): remove unused mysql golden file (#6106) 2022-10-04 20:26:51 -07:00
Aditya Radhakrishnan
ee1a1ef45b
feat(posts): add posts feature to DataHub (#6110) 2022-10-04 15:37:28 -07:00
Harshal Sheth
3c0f63c50a
fix(ingest): hide deprecated path_spec option from config (#5944) 2022-10-04 12:14:00 -07:00
Thomas Tauber-Marshall
6b83cab82c
feat(ingest): add CorpUser and CorpGroup to the Python SDK (#5930) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2022-10-04 12:13:38 -07:00
Gabe Lyons
1325b8ad71
feat(search): improved search snippet FE logic (#6109) 
* starting improvements on search snippet

* flesh out ranking logic

* adding tests

* responding to comments
 2022-10-04 11:27:46 -07:00
Gabe Lyons
ce90310dd0
feat(advanced-search): Complete Advanced Search: backend changes & tying UI together (#6068) 
* stashing progress

* adding remove option

* more progress

* editing

* further in

* additional rendering improvements

* stashing adv search progress

* stashing more progress

* propagating not filters back to UI

* more frontend progress

* more filters working

* getting ready for data platform selector

* add platform select functionality

* locking out switching btwn advanced and standard filters

* final polish

* remove unneeded code

* added unit and cypress tests

* resolutions after merge

* adding documentation

* cleaning up & refactoring

* removing console.log

* minor ui fix & removing unneeded code

* fixing lineage search

* fixing lints

* fix display of degree

* fixing test

* fixing lint

* responding to comments

* fixing tests

* fix smoke tests

* fixing cypress

* fixing cypress test

* responding to comments
 2022-10-04 10:20:04 -07:00
Peter Szalai
396fd31ddc
fix(mae-consumer): use JAVA_TOOL_OPTIONS instead of JDK_JAVA_OPTIONS (#6114) 2022-10-04 09:07:44 -07:00
Roko Gudić
d52fde126f
feat(ingest): salesforce - add sandbox support (#6105) 
Co-authored-by: rgudic <roko.gudic@infobip.com>
 2022-10-04 09:05:53 -07:00
John Joyce
e2c5d2a4ce
feat(ui): bump max recommendations for Platforms (#6113) 2022-10-04 18:35:04 +05:30
Shirshanka Das
bea5a0751e
chore(deps): upgrade opentelemetry dependencies (#6100) 
Co-authored-by: Pedro Silva <pedro@acryl.io>
 2022-10-03 22:38:09 -07:00
RyanHolstien
bfb903cfb8
feat(ingest): add async option to ingest proposal endpoint (#6097) 
* feat(ingest): add async option to ingest proposal endpoint

* small tweak to validate before write to K, also keep existing path for timeseries aspects

* avoid double convert

Co-authored-by: Shirshanka Das <shirshanka@apache.org>
 2022-10-03 19:56:19 -05:00
Peter Szalai
5fb875a32b
feat(docs): Improve documentation on Search (#5889) 2022-10-03 09:35:46 +02:00
Shirshanka Das
e688dd554a
chore(deps): upgrade neo4j to 4.4.x (#6101) 2022-10-02 23:16:45 -07:00
Shirshanka Das
bab5acc5e3
chore(deps): upgrade graphql-java deps to 19.0 (#6099) 2022-10-02 19:35:10 -07:00
Shirshanka Das
d590ae8f2f
docs: add information about mapping ports for datahub-gms (#6092) 2022-10-02 16:47:21 -07:00
Shirshanka Das
48b46971b8
feat(ingest): remove hardcoded env variable default for cli version (#6075) 2022-10-02 16:46:34 -07:00
Tamas Nemeth
05f5c123bc
fix(ingest): bigquery-beta - Lowering a bit memory footprint of bigquery usage (#6095) 
* Lowering a bit memory footprint of bigquery usage
* Filtering out not seen tables from usage generation
 2022-10-01 14:39:04 +02:00
Gabe Lyons
266216311e
fix(lineage): fix missed casing in lineage registry (#6078) 
* fix backwards case sensitive lineage

* update tests

* make EdgeInfo equals and hashcode case resilient

* updating lint

Co-authored-by: John Joyce <john@acryl.io>
 2022-09-30 13:52:28 -07:00
Aseem Bansal
79575b2815
feat(retention): change default policy, add API to apply retention (#6088) 2022-09-30 16:11:39 +05:30
Shirshanka Das
3d8d8c6f8e
fix(ingest): lookml - extract column correctly (#6093) 2022-09-30 10:03:22 +02:00
Harshal Sheth
446b10dcc3
fix(ingest): bump lkml version (#6091) 2022-09-29 20:46:03 -07:00
Gabe Lyons
7359d922fc
feat(advanced search): Add component to show all advanced search filters & add new filter (#6058) 
* adding select value modal

* expanding gql model and adding advanced search filter component

* adding advanced search filters component

* removing console logs

* improve readability

* responding to comments
 2022-09-29 17:25:38 -07:00
Peter Szalai
4792ac56b7
feat(monitoring): track graphql errors in metrics (#6087) 
* feat(monitoring): track graphql errors in metrics

* fix(statuscode): use status code

* address linting

* address linting
 2022-09-29 21:36:17 +02:00
Gabe Lyons
8163e1cbe9
fix(ui): fix schema infinite rerender (#6082) 2022-09-29 08:38:19 -07:00
Tamas Nemeth
a6930562fd
fix(ingest): presto-on-hive - Setting display name properly (#6065) 2022-09-29 16:20:17 +02:00
Harshal Sheth
71f5ec7f19
fix(ingest): mssql - alias for mssql-odbc source (#6080) 2022-09-29 15:33:49 +02:00
Harshal Sheth
05b9d4ef2f
fix(doc): note that Athena doesn't support lineage (#6081) 2022-09-29 15:30:09 +02:00
Tomáš Kubín
596d48448e
fix(elasticsearch) Analytics indices creation on AWS ES (#5502) 
* refactor(elasticsearch-setup-job): create-indices.sh readability

The script contains many copy-pasting and is not easy to follow.
Add comments, extract commonly used operations into functions, unify approaches.

* fix(elasticsearch-setup-job): AWS indices creation

Fix the issue where Amazon OpenSearch (AWS ES) indices are incorrectly initialised
and the Analytics screen shows errors only.

* feat(elasticsearch-setup-job): configuration hint

mention USE_AWS_ELASTICSEARCH env value if it seems it's set the wrong way

* fix(elasticsearch-setup-job): silent curl

* fix(elasticsearch-setup-job): better USE_AWS_ELASTICSEARCH hint

* docs(elasticsearch-setup-job): index dropping explained

- more comments
- more defensive approach
- index file renamed

* fix(elasticsearch-setup-job): script fixes

* merge(elasticsearch-setup-job): merging in PR #5937

* merge(elasticsearch-setup-job): merging in PR #5963

* merge(elasticsearch-setup-job): merging in PR #5975

Co-authored-by: Pedro Silva <pedro@acryl.io>
 2022-09-29 14:10:51 +01:00
Patrick Marx
9e7bd1a84f
fix(mae-consumer): fix regression on base64 encoding (#6061) 
Pull request #5827 introduced a regression by removing coreutils
from the mae-consumer Dockerfile (coreutils was added in #3723).
This broke the base64 call in the startup script s.th. the Elasticsearch
auth header will not be correctly set when username and password are
provided.

To make sure that the startup script fails on these errors in the
future, set "-euo pipefail" which lets the bash script fail on errors
and unset variables.
Also refactor the startup script to make it more stable and readable.

Co-authored-by: Pedro Silva <pedro@acryl.io>
 2022-09-29 12:13:45 +01:00