mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-12-27 23:48:19 +00:00
Pluggable API/Features Limits (#16782)
* Limits * Limits * - Mismatched Types * Update Limits config response * Update Limits feature response * Limits * Limits * - Mismatched Types * Update Limits config response * Update Limits feature response * Limits: add entity resource enforcer * Limits: fix rebase * update limits enforcement * Add OperationContext to limits * chore: Bump versions to `1.4.0` * chore: Bump Ingestion Versions to `1.4.0.1` for Release * chore: Bump Ingestion Versions to `1.4.0.1` in Dockerfiles for Release * Remove Retry From Abstract Event Consumer (#16405) (cherry picked from commit f8ed079731cc238dc136306fe018c5df35dd2f3b) * Fix Migrations: Add postgres migrations (#16403) (cherry picked from commit 9416a7ac5fa8fd9695063b108501790d813e8e6e) * Add Null Check for isAdmin (#16407) * Remove Retry From Abstract Event Consumer * - Add Check for null Or Empty in isAdmin * - Fix Test (cherry picked from commit fe2db2d63c5495b6c288d4252a19ab77481b6de0) * Fix OpenLineage ingestor (#16416) * Fix OpenLineage ingestor * py format --------- Co-authored-by: ulixius9 <mayursingal9@gmail.com> * Minor: added whats new for 1.4.1 (#16420) * Minor: added whats new for 1.4.1 * added note in to whats new * Fix SSL issue (#16412) * chore: Bump Versions for `1.4.1` Release * chore(release): Prepare Branch for `1.4.2` * [MINOR] partition migration issue with redshift servics (#16452) * fix: partition migration issue with redshift servics * chore: typo in sql comment (cherry picked from commit 451d73593e813151c24f2c1d17efb3dcdebb71c8) * minor(ui): update what's new for 1.4.2 (#16457) (cherry picked from commit d55981adfd2321de706e4a043828bb473a4b05f1) * fix: ingestion for dbt > 1.8.0 resource_type is not an enum (#16415) * fix: resource_type is not an enum * feat: add log to display finis * improve readability * use getattr to be compatible * format * Add Cache Query Param for Limits * Only Parse view query (#16470) * add limit check during user creation via PUT * add limit check during user creation via PUT * MINOR: Kafka Setup SSL Arg Fix (#16469) * Fix#16404 - Show Node level lineage by default (#16445) * default to node layer * update cypress * code cleanup * fix cypress (cherry picked from commit f0cda8464f34a21f45f18fa557e980fb2f105d8e) * Invalidate count of data asset after hard delete. add limit exception to ingestion client * - Remove Change Description from Lineage (#16488) (cherry picked from commit 9e5c5529a84dfc781382b3a3b6abd80ee41f11f5) * - Non Indexable fields should be remvoed at the end (#16499) (cherry picked from commit f0b0f7a9426ca601d3bfee3989d4ce47e732a7af) * fix announcement not redirect from landing page (#16506) * fix announcement not redirect from landing page * minor changes * change in cypress test (cherry picked from commit ee7cddd169a3a1fb1e598e80035c2fc15a5a129b) * Fix Schema Field Null Issue (#16510) (cherry picked from commit 022772943f1b33f6230cb35547d1da6acfaf6cfa) * feat(ui): limits integration with application (#16206) * feat(ui): limits integration with application * support pipelineSchedules via limit api * enforce limit to all the modules * update banner styling * update * support disable option for ManageButton * limit version * fix spotlight * update tests * Add name and version history to resource limits Refactor the getEntityIcon function and add new icon mappings * limit version * hide access token tab * fix version for all the entity * fix tests * fix DQ tests * Add fallback for the icon * Revert the fallback icon changes * Apply the limit to the add ingestion button in the service details page * Fix the data quality tab add test button not working * fix banner styling * minor fix * Fix ingestion component unit test * Add InlineAlert component * update entityNameLabels mapping object * Fix the incorrect link in LimitBanner * update pricing page url * Create the GlobalSettingsClassBase * Update URLs for pricing page and upgrade options * fix global settings uncaught error * add parameters to the resource limit API * implement inline alerts for service and alert creation form * update PRIVILEGES for docker * fix layout issues * fix tests --------- Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com> * Add token limitations * Add token limitations * Add appType as part of schema in ingestion pipeline (#16519) * #16489: fix the redirect issue to new tab for tags and glossary (#16512) * fix the redirect issue to new tab for tags and glossary * fix the redirect on cancel icon and unit test issue * changes as per comments (cherry picked from commit 8d312f0853609cfef260739cf789d459838a3421) * Fix #16229 - Tag and Service filters for test cases (#16484) * fix: added test case support for tags (inherit from table/column)]" * feat: add tag and service filter for test cases * feat: add tier query param * fix: tests (cherry picked from commit 6b00dde90285924445567ee7c396c89f0fcf3f1d) * fix: None type is not iterable (#16496) (cherry picked from commit 656da03b14ca24171cf7924b9dd33663e6bed423) * minor(ui): refresh token for OIDC SSO (#16483) * minor(ui): refresh token for OIDC SSO * remove frame window timeout issue * increase iFrame timeout for oidc (cherry picked from commit 1a6c4c972052836a9b3cfa273b7ea1aa3202eafe) * feat(ui): support tag & tier filter for test case (#16502) * feat(ui): support tag & tier filter for test case * fix tag filter * allow single select for tier * added service name filter * update cypress for tags, tier & service * add specific add for filters * fix tier api call (cherry picked from commit 5b71d79e8ac2d08a154882dfe71b9b3a0f73bffc) * minor: sanitize activity feed editor content (#16533) * Add appType as part of schema in ingestion pipeline (#16519) * Fixed quicksight conn (#16537) * fix: saml auth for new user not created (#16543) * fix: saml auth for new user not created * doc: add comment * Fix#16491 - fix lineage edge description update (#16538) * fix lineage edge description update * fix tests (cherry picked from commit dff0aa8dbedcd4064ad63765cadda65bb998772e) * CYPRESS: fix announcement cypress (#16536) * fix announcement cypress * changes as per comments * fix the cypress failure (cherry picked from commit fcb87b5866ba06aa7a6db516677e311c24053db7) * [MINOR] Fix Test Failure for EventRegistration * [MINOR] Fix Test Failure for EventRegistration * [MINOR] Fix Test Failure for EventRegistration [MINOR] Fix Test Failure for EventRegistration * Fix Event Handlers registration Issue (#16544) * Fix Event Handlers Issue * Review Comments (cherry picked from commit d374e48b7938e8ad3514dc5cf8dff619a12595e3) * [MINOR] Fix Test Failure for EventRegistration (cherry picked from commit 4563ad4fd10f9790c21fe744d8fc131ebd028ac8) * Fix Topic Schema missing messageSchema (#16545) (cherry picked from commit b612dd90c07f564d38392b1ccfe0de1505a4867b) * Add limits exception cache in rest client * MINOR: Ignore Cluster Information from columns (#16495) * minor: improve the block editor initial content history (#16540) * Minor: fixed data quality page type issue (#16556) * #16521: fix issue in userProfilePage for roles. teams and displayName (#16527) * fix update on roles and backlink them in user profile page * fix teams, displayName and profile pic issue * sonar fix * fix cypress issue * minor changes (cherry picked from commit 98945cb2db87ebb325d3a72131f049abffcba345) * Empty quick filters (#16402) * initial commit for empty quick filters * update progress * fix field title * cleanup * add tests * unit tests * fix encoding of search query * add cypress tests * add cypress * fix flaky cypress * fix review comments * revert tooltip changes * fix tests * fix tests (cherry picked from commit 5930cd7a7a4bef73f6850848c85118eb64843e2d) * Fix #16278 : Search to display Draft glossaryTerms on Explore page (#16462) * Fix #16278 : Search to display Draft glossaryTerms as well on Explore page * add term status quick filter * change aggregation key for status field * change aggregation key for status field * add lowercase_normalizer in status filed for aggregate api * add cypress tests * fix cypress --------- Co-authored-by: karanh37 <karanh37@gmail.com> Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com> (cherry picked from commit ae5e9d61cc9e6a39d65972987de9149a421395b1) * [FIX] GlossaryTerm reviewers should be user or team only (#16372) * add teams as reviewer * Check Users to be reviewers * Reviewers can be a team or user * Fix check by id or name * Review can be team or user both * Validate Reviewers * add multi select control * - Fix Reviewers * - Centralize Reviewer Relationship to EntityRepository * - Sort * add team as reviewer for glossary terms * locales * cleanup * - Update Reviewer should remove existing reviewers * fix selectable owner control * fix code smells * fix reviewer issue * add glossary cypress * fix patch issue on reviewers set to null * update cypress tests * fix cypress * fix cypress * fix reviewers in glossary task and supported cypress * fix pytest * Fix * fix cypress * fix code smells * Inherited Reviewers need to be present always * filter out inherited users * fix cypress * fix backend tests failure * fix backend tests failure -checkstyle * restrict owner to accept task in case of reviewer present * fix pytest --------- Co-authored-by: karanh37 <karanh37@gmail.com> Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com> Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com> Co-authored-by: Ashish Gupta <ashish@getcollate.io> Co-authored-by: ulixius9 <mayursingal9@gmail.com> Co-authored-by: sonikashah <sonikashah94@gmail.com> (cherry picked from commit 9ec3d94e3b8445e63a7d77239c92c92a32536bf2) * Add testSuite tags, domain field and check for TestCase limits * fix owner not showing after refersh in teams page (#16567) (cherry picked from commit 119fcf8959732a980b75e1f795a9f2dc5288cd27) * [ISSUE-16503] Fix createUser to use EntityResource (#16549) * Fix createUser to use EntityResource * fix broken tests * Fix Tests - 3 (cherry picked from commit aeb020ae3b0cbab3a2ee5995c61480cdd1eae405) * what's new for 1.4.2 (#16568) (cherry picked from commit c86468d9929e433922886852381269b46d69c832) * address feedbacks * fix error for bots page * update banner text * allow force fetch limit * fix ingestion schedule * Revert "Merge branch '1.4.2' into limits" This reverts commit 8e965207a23ba527d0f5ba91463c1869077bf091, reversing changes made to 4d16531965fb0d489a4afdebd45ab5b7f3d1eb5c. * Merge 1.4.2 (#16578) * fix explore page conflicts * fix tests --------- Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com> Co-authored-by: Chira Madlani <chirag@getcollate.io> * fix subheader * Updating glossary reviewers should propagate reviewers in glossary term (#16580) * highlight inherited reviewer in glossary * locales * use glossary name for search query * fix glossary version cypress * add union datatype for subfields * Adding reviewer to glossary also adds them as an assignee to the task * add glossary approval cypress --------- Co-authored-by: sonikashah <sonikashah94@gmail.com> (cherry picked from commit 4c8bf1cac14074df87dafe7a719e2795b0a29895) * Update documentation for Search Index apis (#16539) (cherry picked from commit d3123c49143652015c416d271d9fd0f9cfa9e324) * cypress: fixed flakiness and announcment cypress (#16579) * fetch latest limit for create / delete operations * guard datAsset limit got topic, dashboard, mlmodel etc * Fix: Ensure correct index mapping in Elasticsearch for clusterAlias (#16589) * Fix: Ensure correct index mapping in Elasticsearch for clusterAlias * Fix: Ensure correct index mapping in Elasticsearch for clusterAlias (cherry picked from commit 8723b8c36afe31410c31d1ebbdafe7b1770921fa) * cypress: fixed cypress AUT for mysql (#16446) * cypress: fixed cypress AUT for mysql * minor fix * skip announcment redirection cypress * Minor: Ensure correct index mapping in Elasticsearch for clusterAlias (#16598) (cherry picked from commit 04543722a6f6e2b1eaf7a451ebb1c176862bc346) * Fix Postgres Application listing (#16600) * Fix Postgres Application listing * Fix Listing (cherry picked from commit 77dfe1f6af53d187ff7a61fdb1e1416de7178f5a) * fix limit related issue * Fix Automations limits invalidation during the uninstall * cypress: fixed 1.4.2 AUT cypress (#16602) * cypress: fixed 1.4.2 AUT cypress * fix cypress around announcement,user,glossary, lineage and mydata * searchIndexApplication fix and minor changes --------- Co-authored-by: Ashish Gupta <ashish@getcollate.io> * test: add updateJWTTokenExpiryTime util (#16606) (cherry picked from commit 8c173bed6a279cb0a648bd30632ea6ebdf4a2a90) * OSS changes for adding automator cypress tests (#16611) * Fix Test Suite Filter (#16615) Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com> (cherry picked from commit 3db41f08e27f388495040e5b23cc7bee5ae665f1) * MINOR: Fix Profiler for SSL Enabled Source (#16613) * Add Test Suite SSL (#16619) * MINOR: Fix ssl connection in usage & lineage (#16625) * Fix owner notification (#16629) * - Fix Task notification not getting sent to owners * - Fix Task notification not getting sent to owners (cherry picked from commit cc2d581eb0524604b6dcf0523e9ca96e0b8a6ce3) * chore(release): Prepare Branch for `1.4.3` * - Fix User Signup (#16667) (cherry picked from commit b4cba8a850ecd7a25aeff6ca7dea0dc432d43d86) * - Fix User Signup - p2 (cherry picked from commit d9ae6f6db9891f8e9bf7ad49c561a71dd50103da) * - Update What's new (#16669) - fix vulnerability (cherry picked from commit 1dcb1bd46f9da49764f4c61a7ac5048dd2fa956b) * Minor: Fix incorrect alert on signup page (#16666) * Fix Application enforceLimits during install * Wrap the add test button with limits wrapper for column profile tab * fix errors * fix tests * fix pylint * fix tests * fix limits * pylint * fix schedule options * fix glossary spec failure * Add domain & tags to testSuite * Update airflow-apis-tests-3_9.yml --------- Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com> Co-authored-by: Chira Madlani <chirag@getcollate.io> Co-authored-by: Pablo Takara <pjt1991@gmail.com> Co-authored-by: Akash-Jain <15995028+akash-jain-10@users.noreply.github.com> Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com> Co-authored-by: Ayush Shah <ayush@getcollate.io> Co-authored-by: Maxim Martynov <martinov_m_s_@mail.ru> Co-authored-by: ulixius9 <mayursingal9@gmail.com> Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com> Co-authored-by: Teddy <teddy.crepineau@gmail.com> Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com> Co-authored-by: Antoine Balliet <antoine.balliet@gorgias.com> Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com> Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com> Co-authored-by: Ashish Gupta <ashish@getcollate.io> Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com> Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com> Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com> Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com> Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com> Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Sriharsha Chintalapani
GitHub
parent
bf1dddedeb
commit
5571851e53
2
.github/workflows/airflow-apis-tests-3_9.yml
vendored
2
.github/workflows/airflow-apis-tests-3_9.yml
vendored
@ -100,7 +100,7 @@ jobs:
|
||||
- name: Start Server and Ingest Sample Data
|
||||
uses: nick-fields/retry@v2.8.3
|
||||
env:
|
||||
INGESTION_DEPENDENCY: "mysql,elasticsearch"
|
||||
INGESTION_DEPENDENCY: "mysql,elasticsearch,sample-data"
|
||||
with:
|
||||
timeout_minutes: 60
|
||||
max_attempts: 2
|
||||
|
||||
@ -18,7 +18,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<properties>
|
||||
|
||||
@ -370,6 +370,11 @@ email:
|
||||
password: ${SMTP_SERVER_PWD:-""}
|
||||
transportationStrategy: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}
|
||||
|
||||
limits:
|
||||
enable: ${LIMITS_ENABLED:-false}
|
||||
className: ${LIMITS_CLASS_NAME:-"org.openmetadata.service.limits.DefaultLimits"}
|
||||
limitsConfigFile: ${LIMITS_CONFIG_FILE:-""}
|
||||
|
||||
web:
|
||||
uriPath: ${WEB_CONF_URI_PATH:-"/api"}
|
||||
hsts:
|
||||
|
||||
@ -18,7 +18,7 @@ volumes:
|
||||
services:
|
||||
ingestion:
|
||||
container_name: openmetadata_ingestion
|
||||
image: docker.getcollate.io/openmetadata/ingestion:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/ingestion:1.4.3
|
||||
environment:
|
||||
AIRFLOW__API__AUTH_BACKENDS: "airflow.api.auth.backend.basic_auth,airflow.api.auth.backend.session"
|
||||
AIRFLOW__CORE__EXECUTOR: LocalExecutor
|
||||
|
||||
@ -14,7 +14,7 @@ services:
|
||||
execute-migrate-all:
|
||||
container_name: execute_migrate_all
|
||||
command: "./bootstrap/openmetadata-ops.sh migrate"
|
||||
image: docker.getcollate.io/openmetadata/server:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/server:1.4.3
|
||||
environment:
|
||||
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
|
||||
SERVER_PORT: ${SERVER_PORT:-8585}
|
||||
@ -227,7 +227,7 @@ services:
|
||||
openmetadata-server:
|
||||
container_name: openmetadata_server
|
||||
restart: always
|
||||
image: docker.getcollate.io/openmetadata/server:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/server:1.4.3
|
||||
environment:
|
||||
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
|
||||
SERVER_PORT: ${SERVER_PORT:-8585}
|
||||
|
||||
@ -11,7 +11,7 @@
|
||||
|
||||
# Build stage
|
||||
FROM alpine:3.19 AS build
|
||||
ARG RI_VERSION="1.5.0-SNAPSHOT"
|
||||
ARG RI_VERSION="1.4.3"
|
||||
ENV RELEASE_URL="https://github.com/open-metadata/OpenMetadata/releases/download/${RI_VERSION}-release/openmetadata-${RI_VERSION}.tar.gz"
|
||||
|
||||
RUN mkdir -p /opt/openmetadata && \
|
||||
@ -21,7 +21,7 @@ RUN mkdir -p /opt/openmetadata && \
|
||||
|
||||
# Final stage
|
||||
FROM alpine:3.19
|
||||
ARG RI_VERSION="1.5.0-SNAPSHOT"
|
||||
ARG RI_VERSION="1.4.3"
|
||||
ARG BUILD_DATE
|
||||
ARG COMMIT_ID
|
||||
LABEL maintainer="OpenMetadata"
|
||||
|
||||
@ -18,7 +18,7 @@ volumes:
|
||||
services:
|
||||
postgresql:
|
||||
container_name: openmetadata_postgresql
|
||||
image: docker.getcollate.io/openmetadata/postgresql:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/postgresql:1.4.3
|
||||
restart: always
|
||||
command: "--work_mem=10MB"
|
||||
environment:
|
||||
@ -61,7 +61,7 @@ services:
|
||||
|
||||
execute-migrate-all:
|
||||
container_name: execute_migrate_all
|
||||
image: docker.getcollate.io/openmetadata/server:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/server:1.4.3
|
||||
command: "./bootstrap/openmetadata-ops.sh migrate"
|
||||
environment:
|
||||
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
|
||||
@ -275,7 +275,7 @@ services:
|
||||
openmetadata-server:
|
||||
container_name: openmetadata_server
|
||||
restart: always
|
||||
image: docker.getcollate.io/openmetadata/server:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/server:1.4.3
|
||||
environment:
|
||||
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
|
||||
SERVER_PORT: ${SERVER_PORT:-8585}
|
||||
@ -483,7 +483,7 @@ services:
|
||||
|
||||
ingestion:
|
||||
container_name: openmetadata_ingestion
|
||||
image: docker.getcollate.io/openmetadata/ingestion:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/ingestion:1.4.3
|
||||
depends_on:
|
||||
elasticsearch:
|
||||
condition: service_started
|
||||
|
||||
@ -18,7 +18,7 @@ volumes:
|
||||
services:
|
||||
mysql:
|
||||
container_name: openmetadata_mysql
|
||||
image: docker.getcollate.io/openmetadata/db:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/db:1.4.3
|
||||
command: "--sort_buffer_size=10M"
|
||||
restart: always
|
||||
environment:
|
||||
@ -59,7 +59,7 @@ services:
|
||||
|
||||
execute-migrate-all:
|
||||
container_name: execute_migrate_all
|
||||
image: docker.getcollate.io/openmetadata/server:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/server:1.4.3
|
||||
command: "./bootstrap/openmetadata-ops.sh migrate"
|
||||
environment:
|
||||
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
|
||||
@ -273,7 +273,7 @@ services:
|
||||
openmetadata-server:
|
||||
container_name: openmetadata_server
|
||||
restart: always
|
||||
image: docker.getcollate.io/openmetadata/server:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/server:1.4.3
|
||||
environment:
|
||||
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
|
||||
SERVER_PORT: ${SERVER_PORT:-8585}
|
||||
@ -482,7 +482,7 @@ services:
|
||||
|
||||
ingestion:
|
||||
container_name: openmetadata_ingestion
|
||||
image: docker.getcollate.io/openmetadata/ingestion:1.5.0-SNAPSHOT
|
||||
image: docker.getcollate.io/openmetadata/ingestion:1.4.3
|
||||
depends_on:
|
||||
elasticsearch:
|
||||
condition: service_started
|
||||
|
||||
@ -5,5 +5,6 @@ CREATE USER 'airflow_user'@'%' IDENTIFIED BY 'airflow_pass';
|
||||
GRANT ALL PRIVILEGES ON openmetadata_db.* TO 'openmetadata_user'@'%' WITH GRANT OPTION;
|
||||
GRANT PROCESS, USAGE ON *.* TO 'openmetadata_user'@'%';
|
||||
GRANT ALL PRIVILEGES ON airflow_db.* TO 'airflow_user'@'%' WITH GRANT OPTION;
|
||||
GRANT PROCESS, USAGE ON *.* TO 'openmetadata_user'@'%';
|
||||
FLUSH PRIVILEGES;
|
||||
commit;
|
||||
|
||||
@ -68,7 +68,7 @@ ARG INGESTION_DEPENDENCY="all"
|
||||
ENV PIP_NO_CACHE_DIR=1
|
||||
# Make pip silent
|
||||
ENV PIP_QUIET=1
|
||||
ARG RI_VERSION="1.5.0.0.dev0"
|
||||
ARG RI_VERSION="1.4.3.0"
|
||||
RUN pip install --upgrade pip
|
||||
RUN pip install "openmetadata-managed-apis~=${RI_VERSION}" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.7.3/constraints-3.10.txt"
|
||||
RUN pip install "openmetadata-ingestion[${INGESTION_DEPENDENCY}]~=${RI_VERSION}"
|
||||
|
||||
@ -73,7 +73,7 @@ ENV PIP_QUIET=1
|
||||
RUN pip install --upgrade pip
|
||||
|
||||
ARG INGESTION_DEPENDENCY="all"
|
||||
ARG RI_VERSION="1.5.0.0.dev0"
|
||||
ARG RI_VERSION="1.4.3.0"
|
||||
RUN pip install --upgrade pip
|
||||
RUN pip install "openmetadata-ingestion[airflow]~=${RI_VERSION}"
|
||||
RUN pip install "openmetadata-ingestion[${INGESTION_DEPENDENCY}]~=${RI_VERSION}"
|
||||
|
||||
@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
|
||||
# since it helps us organize and isolate version management
|
||||
[project]
|
||||
name = "openmetadata-ingestion"
|
||||
version = "1.5.0.0.dev0"
|
||||
version = "1.4.3.0"
|
||||
dynamic = ["readme", "dependencies", "optional-dependencies"]
|
||||
authors = [
|
||||
{name = "OpenMetadata Committers"}
|
||||
|
||||
@ -131,11 +131,10 @@ class TableDiffParamsSetter(RuntimeParameterSetter):
|
||||
@staticmethod
|
||||
def get_data_diff_url(service_url: str, table_fqn) -> str:
|
||||
url = urlparse(service_url)
|
||||
# remove the drivername from the url becuase table-diff doesn't support it
|
||||
# remove the driver name from the url because table-diff doesn't support it
|
||||
kwargs = {"scheme": url.scheme.split("+")[0]}
|
||||
service, database, schema, table = fqn.split( # pylint: disable=unused-variable
|
||||
table_fqn
|
||||
)
|
||||
# pylint: disable=unbalanced-tuple-unpacking
|
||||
_, database, schema, _ = fqn.split(table_fqn)
|
||||
# path needs to include the database AND schema in some of the connectors
|
||||
if kwargs["scheme"] in ["mssql"]:
|
||||
kwargs["path"] = f"/{database}/{schema}"
|
||||
@ -143,9 +142,8 @@ class TableDiffParamsSetter(RuntimeParameterSetter):
|
||||
|
||||
@staticmethod
|
||||
def get_data_diff_table_path(table_fqn: str):
|
||||
service, database, schema, table = fqn.split( # pylint: disable=unused-variable
|
||||
table_fqn
|
||||
)
|
||||
# pylint: disable=unbalanced-tuple-unpacking
|
||||
_, _, schema, table = fqn.split(table_fqn)
|
||||
return fqn._build( # pylint: disable=protected-access
|
||||
"___SERVICE___", "__DATABASE__", schema, table
|
||||
).replace("___SERVICE___.__DATABASE__.", "")
|
||||
|
||||
@ -21,6 +21,7 @@ from requests.exceptions import HTTPError
|
||||
|
||||
from metadata.config.common import ConfigModel
|
||||
from metadata.ingestion.ometa.credentials import URL, get_api_version
|
||||
from metadata.ingestion.ometa.ttl_cache import TTLCache
|
||||
from metadata.utils.execution_time_tracker import calculate_execution_time
|
||||
from metadata.utils.logger import ometa_logger
|
||||
|
||||
@ -33,6 +34,12 @@ class RetryException(Exception):
|
||||
"""
|
||||
|
||||
|
||||
class LimitsException(Exception):
|
||||
"""
|
||||
API Client Feature Limit exception
|
||||
"""
|
||||
|
||||
|
||||
class APIError(Exception):
|
||||
"""
|
||||
Represent API related error.
|
||||
@ -97,7 +104,8 @@ class ClientConfig(ConfigModel):
|
||||
api_version: Optional[str] = "v1"
|
||||
retry: Optional[int] = 3
|
||||
retry_wait: Optional[int] = 30
|
||||
retry_codes: List[int] = [429, 504]
|
||||
limit_codes: List[int] = [429]
|
||||
retry_codes: List[int] = [504]
|
||||
auth_token: Optional[Callable] = None
|
||||
access_token: Optional[str] = None
|
||||
expires_in: Optional[int] = None
|
||||
@ -107,8 +115,10 @@ class ClientConfig(ConfigModel):
|
||||
allow_redirects: Optional[bool] = False
|
||||
auth_token_mode: Optional[str] = "Bearer"
|
||||
verify: Optional[Union[bool, str]] = None
|
||||
ttl_cache: int = 60
|
||||
|
||||
|
||||
# pylint: disable=too-many-instance-attributes
|
||||
class REST:
|
||||
"""
|
||||
REST client wrapper to manage requests with
|
||||
@ -124,10 +134,13 @@ class REST:
|
||||
self._retry = self.config.retry
|
||||
self._retry_wait = self.config.retry_wait
|
||||
self._retry_codes = self.config.retry_codes
|
||||
self._limit_codes = self.config.limit_codes
|
||||
self._auth_token = self.config.auth_token
|
||||
self._auth_token_mode = self.config.auth_token_mode
|
||||
self._verify = self.config.verify
|
||||
|
||||
self._limits_reached = TTLCache(config.ttl_cache)
|
||||
|
||||
def _request( # pylint: disable=too-many-arguments
|
||||
self,
|
||||
method,
|
||||
@ -139,6 +152,9 @@ class REST:
|
||||
headers: dict = None,
|
||||
):
|
||||
# pylint: disable=too-many-locals
|
||||
if path in self._limits_reached:
|
||||
raise LimitsException(f"Skipping request - limits reached for {path}")
|
||||
|
||||
if not headers:
|
||||
headers = {"Content-type": "application/json"}
|
||||
base_url = base_url or self._base_url
|
||||
@ -194,6 +210,10 @@ class REST:
|
||||
while retry >= 0:
|
||||
try:
|
||||
return self._one_request(method, url, opts, retry)
|
||||
except LimitsException as exc:
|
||||
logger.error(f"Feature limit exceeded for {url}")
|
||||
self._limits_reached.add(path)
|
||||
raise exc
|
||||
except RetryException:
|
||||
retry_wait = self._retry_wait * (total_retries - retry + 1)
|
||||
logger.warning(
|
||||
@ -217,6 +237,7 @@ class REST:
|
||||
Returns the body json in the 200 status.
|
||||
"""
|
||||
retry_codes = self._retry_codes
|
||||
limit_codes = self._limit_codes
|
||||
try:
|
||||
resp = self._session.request(method, url, **opts)
|
||||
resp.raise_for_status()
|
||||
@ -234,6 +255,8 @@ class REST:
|
||||
# retry if we hit Rate Limit
|
||||
if resp.status_code in retry_codes and retry > 0:
|
||||
raise RetryException() from http_error
|
||||
if resp.status_code in limit_codes:
|
||||
raise LimitsException() from http_error
|
||||
if "code" in resp.text:
|
||||
error = resp.json()
|
||||
if "code" in error:
|
||||
|
||||
47
ingestion/src/metadata/ingestion/ometa/ttl_cache.py
Normal file
47
ingestion/src/metadata/ingestion/ometa/ttl_cache.py
Normal file
@ -0,0 +1,47 @@
|
||||
# Copyright 2021 Collate
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""
|
||||
Simple dictionary implementation for keys with TTL
|
||||
"""
|
||||
from datetime import datetime
|
||||
from typing import Dict
|
||||
|
||||
|
||||
class TTLCache:
|
||||
"""
|
||||
class to handle ttl cache
|
||||
"""
|
||||
|
||||
def __init__(self, ttl: int):
|
||||
self._ttl = ttl
|
||||
# The key will be the object, and the value the created time to check the TTL
|
||||
self._cache: Dict[str, int] = {}
|
||||
|
||||
@staticmethod
|
||||
def _now() -> int:
|
||||
return int(datetime.now().timestamp())
|
||||
|
||||
def __contains__(self, item) -> bool:
|
||||
if item in self._cache:
|
||||
created_at = self._cache[item]
|
||||
if self._now() - created_at > self._ttl:
|
||||
self.delete(item)
|
||||
return False
|
||||
return True
|
||||
return False
|
||||
|
||||
def add(self, value: str):
|
||||
if value not in self._cache:
|
||||
self._cache[value] = self._now()
|
||||
|
||||
def delete(self, key):
|
||||
self._cache.pop(key, None)
|
||||
84
ingestion/tests/unit/test_ttl_cache.py
Normal file
84
ingestion/tests/unit/test_ttl_cache.py
Normal file
@ -0,0 +1,84 @@
|
||||
# Copyright 2021 Collate
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""
|
||||
Test TTL Cache
|
||||
"""
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from metadata.generated.schema.entity.data.table import Table
|
||||
from metadata.generated.schema.entity.services.connections.metadata.openMetadataConnection import (
|
||||
AuthProvider,
|
||||
OpenMetadataConnection,
|
||||
)
|
||||
from metadata.generated.schema.security.client.openMetadataJWTClientConfig import (
|
||||
OpenMetadataJWTClientConfig,
|
||||
)
|
||||
from metadata.ingestion.models.custom_pydantic import CustomSecretStr
|
||||
from metadata.ingestion.ometa.client import REST, LimitsException
|
||||
from metadata.ingestion.ometa.ometa_api import OpenMetadata
|
||||
from metadata.ingestion.ometa.ttl_cache import TTLCache
|
||||
|
||||
|
||||
def test_ttl_cache():
|
||||
"""Check we can add objects to the cache and they expire after the TTL"""
|
||||
|
||||
cache = TTLCache(ttl=1)
|
||||
|
||||
# Set a value
|
||||
cache.add("test")
|
||||
assert "test" in cache
|
||||
|
||||
# Delete the value
|
||||
cache.delete("test")
|
||||
assert "test" not in cache
|
||||
|
||||
# Set a value
|
||||
cache.add("test")
|
||||
assert "test" in cache
|
||||
|
||||
# Wait for the TTL to expire
|
||||
time.sleep(5)
|
||||
|
||||
# Value should not be in the cache
|
||||
assert "test" not in cache
|
||||
|
||||
|
||||
def test_ometa_ttl_cache():
|
||||
"""ometa works well with TTL Cache"""
|
||||
|
||||
server_config = OpenMetadataConnection(
|
||||
hostPort="http://localhost:8585/api",
|
||||
enableVersionValidation=False,
|
||||
authProvider=AuthProvider.openmetadata,
|
||||
securityConfig=OpenMetadataJWTClientConfig(jwtToken=CustomSecretStr("token")),
|
||||
)
|
||||
metadata = OpenMetadata(server_config)
|
||||
|
||||
def limits_exc(*_, **__):
|
||||
raise LimitsException("Mock limits exception")
|
||||
|
||||
with patch.object(REST, "_one_request", side_effect=limits_exc):
|
||||
with pytest.raises(LimitsException) as exc_info:
|
||||
metadata.get_by_name(entity=Table, fqn="random")
|
||||
|
||||
assert str(exc_info.value) == "Mock limits exception"
|
||||
|
||||
with pytest.raises(LimitsException) as exc_info:
|
||||
metadata.get_by_name(entity=Table, fqn="random")
|
||||
|
||||
assert (
|
||||
str(exc_info.value)
|
||||
== "Skipping request - limits reached for /tables/name/random"
|
||||
)
|
||||
@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
|
||||
# since it helps us organize and isolate version management
|
||||
[project]
|
||||
name = "openmetadata_managed_apis"
|
||||
version = "1.5.0.0.dev0"
|
||||
version = "1.4.3.0"
|
||||
readme = "README.md"
|
||||
authors = [
|
||||
{name = "OpenMetadata Committers"}
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>openmetadata-clients</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
|
||||
@ -20,7 +20,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>openmetadata-dist</artifactId>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<artifactId>openmetadata-service</artifactId>
|
||||
|
||||
@ -66,6 +66,7 @@ import org.jdbi.v3.sqlobject.SqlObjects;
|
||||
import org.openmetadata.schema.api.security.AuthenticationConfiguration;
|
||||
import org.openmetadata.schema.api.security.AuthorizerConfiguration;
|
||||
import org.openmetadata.schema.api.security.ClientType;
|
||||
import org.openmetadata.schema.configuration.LimitsConfiguration;
|
||||
import org.openmetadata.schema.services.connections.metadata.AuthProvider;
|
||||
import org.openmetadata.service.apps.ApplicationHandler;
|
||||
import org.openmetadata.service.apps.scheduler.AppScheduler;
|
||||
@ -85,6 +86,8 @@ import org.openmetadata.service.jdbi3.EntityRepository;
|
||||
import org.openmetadata.service.jdbi3.MigrationDAO;
|
||||
import org.openmetadata.service.jdbi3.locator.ConnectionAwareAnnotationSqlLocator;
|
||||
import org.openmetadata.service.jdbi3.locator.ConnectionType;
|
||||
import org.openmetadata.service.limits.DefaultLimits;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.migration.Migration;
|
||||
import org.openmetadata.service.migration.MigrationValidationClient;
|
||||
import org.openmetadata.service.migration.api.MigrationWorkflow;
|
||||
@ -132,6 +135,7 @@ import org.quartz.SchedulerException;
|
||||
public class OpenMetadataApplication extends Application<OpenMetadataApplicationConfig> {
|
||||
private Authorizer authorizer;
|
||||
private AuthenticatorHandler authenticatorHandler;
|
||||
private Limits limits;
|
||||
|
||||
protected Jdbi jdbi;
|
||||
|
||||
@ -202,6 +206,9 @@ public class OpenMetadataApplication extends Application<OpenMetadataApplication
|
||||
// Register Authenticator
|
||||
registerAuthenticator(catalogConfig);
|
||||
|
||||
// Register Limits
|
||||
registerLimits(catalogConfig);
|
||||
|
||||
// Unregister dropwizard default exception mappers
|
||||
((DefaultServerFactory) catalogConfig.getServerFactory())
|
||||
.setRegisterDefaultExceptionMappers(false);
|
||||
@ -526,6 +533,26 @@ public class OpenMetadataApplication extends Application<OpenMetadataApplication
|
||||
}
|
||||
}
|
||||
|
||||
private void registerLimits(OpenMetadataApplicationConfig serverConfig)
|
||||
throws NoSuchMethodException,
|
||||
ClassNotFoundException,
|
||||
IllegalAccessException,
|
||||
InvocationTargetException,
|
||||
InstantiationException {
|
||||
LimitsConfiguration limitsConfiguration = serverConfig.getLimitsConfiguration();
|
||||
if (limitsConfiguration != null) {
|
||||
limits =
|
||||
Class.forName(limitsConfiguration.getClassName())
|
||||
.asSubclass(Limits.class)
|
||||
.getConstructor()
|
||||
.newInstance();
|
||||
} else {
|
||||
LOG.info("Limits config not set, setting DefaultLimits");
|
||||
limits = new DefaultLimits();
|
||||
}
|
||||
limits.init(serverConfig, jdbi);
|
||||
}
|
||||
|
||||
private void registerEventFilter(
|
||||
OpenMetadataApplicationConfig catalogConfig, Environment environment) {
|
||||
if (catalogConfig.getEventHandlerConfiguration() != null) {
|
||||
@ -552,7 +579,7 @@ public class OpenMetadataApplication extends Application<OpenMetadataApplication
|
||||
OpenMetadataApplicationConfig config, Environment environment, Jdbi jdbi) {
|
||||
CollectionRegistry.initialize();
|
||||
CollectionRegistry.getInstance()
|
||||
.registerResources(jdbi, environment, config, authorizer, authenticatorHandler);
|
||||
.registerResources(jdbi, environment, config, authorizer, authenticatorHandler, limits);
|
||||
environment.jersey().register(new JsonPatchProvider());
|
||||
OMErrorPageHandler eph = new OMErrorPageHandler(config.getWebConfiguration());
|
||||
eph.addErrorPage(Response.Status.NOT_FOUND.getStatusCode(), "/");
|
||||
|
||||
@ -31,6 +31,7 @@ import org.openmetadata.schema.api.fernet.FernetConfiguration;
|
||||
import org.openmetadata.schema.api.security.AuthenticationConfiguration;
|
||||
import org.openmetadata.schema.api.security.AuthorizerConfiguration;
|
||||
import org.openmetadata.schema.api.security.jwt.JWTTokenConfiguration;
|
||||
import org.openmetadata.schema.configuration.LimitsConfiguration;
|
||||
import org.openmetadata.schema.email.SmtpSettings;
|
||||
import org.openmetadata.schema.security.secrets.SecretsManagerConfiguration;
|
||||
import org.openmetadata.schema.service.configuration.elasticsearch.ElasticSearchConfiguration;
|
||||
@ -116,6 +117,9 @@ public class OpenMetadataApplicationConfig extends Configuration {
|
||||
@JsonProperty("applications")
|
||||
private AppsPrivateConfiguration appsPrivateConfiguration;
|
||||
|
||||
@JsonProperty("limits")
|
||||
private LimitsConfiguration limitsConfiguration;
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "catalogConfig{"
|
||||
|
||||
@ -0,0 +1,12 @@
|
||||
package org.openmetadata.service.exception;
|
||||
|
||||
import javax.ws.rs.core.Response;
|
||||
import org.openmetadata.sdk.exception.WebServiceException;
|
||||
|
||||
public class LimitsException extends WebServiceException {
|
||||
private static final String ERROR_TYPE = "LIMITS_EXCEPTION";
|
||||
|
||||
public LimitsException(String message) {
|
||||
super(Response.Status.TOO_MANY_REQUESTS, ERROR_TYPE, message);
|
||||
}
|
||||
}
|
||||
@ -138,6 +138,7 @@ public class TestSuiteRepository extends EntityRepository<TestSuite> {
|
||||
Table table =
|
||||
Entity.getEntity(TABLE, testSuite.getExecutableEntityReference().getId(), "owner", ALL);
|
||||
inheritOwner(testSuite, fields, table);
|
||||
inheritDomain(testSuite, fields, table);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -0,0 +1,46 @@
|
||||
package org.openmetadata.service.limits;
|
||||
|
||||
import javax.ws.rs.core.Response;
|
||||
import javax.ws.rs.core.SecurityContext;
|
||||
import org.jdbi.v3.core.Jdbi;
|
||||
import org.openmetadata.schema.configuration.LimitsConfiguration;
|
||||
import org.openmetadata.schema.system.LimitsConfig;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.security.policyevaluator.OperationContext;
|
||||
import org.openmetadata.service.security.policyevaluator.ResourceContextInterface;
|
||||
|
||||
public class DefaultLimits implements Limits {
|
||||
private OpenMetadataApplicationConfig serverConfig = null;
|
||||
private LimitsConfiguration limitsConfiguration = null;
|
||||
private Jdbi jdbi = null;
|
||||
|
||||
@Override
|
||||
public void init(OpenMetadataApplicationConfig serverConfig, Jdbi jdbi) {
|
||||
this.serverConfig = serverConfig;
|
||||
this.limitsConfiguration = serverConfig.getLimitsConfiguration();
|
||||
this.jdbi = jdbi;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void enforceLimits(
|
||||
SecurityContext securityContext,
|
||||
ResourceContextInterface resourceContext,
|
||||
OperationContext operationContext) {
|
||||
// do not enforce limits
|
||||
}
|
||||
|
||||
@Override
|
||||
public LimitsConfig getLimitsConfig() {
|
||||
LimitsConfig limitsConfig = new LimitsConfig();
|
||||
limitsConfig.setEnable(limitsConfiguration.getEnable());
|
||||
return limitsConfig;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Response getLimitsForaFeature(String name, boolean cache) {
|
||||
return Response.ok().build();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void invalidateCache(String entityType) {}
|
||||
}
|
||||
@ -0,0 +1,24 @@
|
||||
package org.openmetadata.service.limits;
|
||||
|
||||
import javax.ws.rs.core.Response;
|
||||
import javax.ws.rs.core.SecurityContext;
|
||||
import org.jdbi.v3.core.Jdbi;
|
||||
import org.openmetadata.schema.system.LimitsConfig;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.security.policyevaluator.OperationContext;
|
||||
import org.openmetadata.service.security.policyevaluator.ResourceContextInterface;
|
||||
|
||||
public interface Limits {
|
||||
void init(OpenMetadataApplicationConfig serverConfig, Jdbi jdbi);
|
||||
|
||||
void enforceLimits(
|
||||
SecurityContext securityContext,
|
||||
ResourceContextInterface resourceContext,
|
||||
OperationContext operationContext);
|
||||
|
||||
LimitsConfig getLimitsConfig();
|
||||
|
||||
Response getLimitsForaFeature(String entityType, boolean cache);
|
||||
|
||||
void invalidateCache(String entityType);
|
||||
}
|
||||
@ -25,7 +25,6 @@ import java.lang.reflect.InvocationTargetException;
|
||||
import java.lang.reflect.Method;
|
||||
import java.net.URI;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.Comparator;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
@ -40,6 +39,7 @@ import org.openmetadata.schema.Function;
|
||||
import org.openmetadata.schema.type.CollectionDescriptor;
|
||||
import org.openmetadata.schema.type.CollectionInfo;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
import org.openmetadata.service.security.auth.AuthenticatorHandler;
|
||||
import org.openmetadata.service.util.ReflectionUtil;
|
||||
@ -88,10 +88,6 @@ public final class CollectionRegistry {
|
||||
}
|
||||
}
|
||||
|
||||
public Map<String, CollectionDetails> getCollectionMap() {
|
||||
return Collections.unmodifiableMap(collectionMap);
|
||||
}
|
||||
|
||||
/**
|
||||
* REST collections are described using *CollectionDescriptor.json Load all CollectionDescriptors from these files in
|
||||
* the classpath
|
||||
@ -152,19 +148,20 @@ public final class CollectionRegistry {
|
||||
Environment environment,
|
||||
OpenMetadataApplicationConfig config,
|
||||
Authorizer authorizer,
|
||||
AuthenticatorHandler authenticatorHandler) {
|
||||
AuthenticatorHandler authenticatorHandler,
|
||||
Limits limits) {
|
||||
// Build list of ResourceDescriptors
|
||||
for (Map.Entry<String, CollectionDetails> e : collectionMap.entrySet()) {
|
||||
CollectionDetails details = e.getValue();
|
||||
String resourceClass = details.resourceClass;
|
||||
try {
|
||||
Object resource =
|
||||
createResource(jdbi, resourceClass, config, authorizer, authenticatorHandler);
|
||||
createResource(jdbi, resourceClass, config, authorizer, authenticatorHandler, limits);
|
||||
details.setResource(resource);
|
||||
environment.jersey().register(resource);
|
||||
LOG.info("Registering {} with order {}", resourceClass, details.order);
|
||||
} catch (Exception ex) {
|
||||
LOG.warn("Failed to create resource for class {} {}", resourceClass, ex);
|
||||
LOG.warn("Failed to create resource for class {} {}", resourceClass, ex.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
@ -181,6 +178,7 @@ public final class CollectionRegistry {
|
||||
OpenMetadataApplicationConfig config,
|
||||
Authorizer authorizer,
|
||||
AuthenticatorHandler authenticatorHandler,
|
||||
Limits limits,
|
||||
boolean isOperations) {
|
||||
// Build list of ResourceDescriptors
|
||||
for (Map.Entry<String, CollectionDetails> e : collectionMap.entrySet()) {
|
||||
@ -188,8 +186,7 @@ public final class CollectionRegistry {
|
||||
if (!isOperations || (isOperations && details.requiredForOps)) {
|
||||
String resourceClass = details.resourceClass;
|
||||
try {
|
||||
Object resource =
|
||||
createResource(jdbi, resourceClass, config, authorizer, authenticatorHandler);
|
||||
createResource(jdbi, resourceClass, config, authorizer, authenticatorHandler, limits);
|
||||
} catch (Exception ex) {
|
||||
LOG.warn("Failed to create resource for class {} {}", resourceClass, ex);
|
||||
}
|
||||
@ -241,7 +238,8 @@ public final class CollectionRegistry {
|
||||
String resourceClass,
|
||||
OpenMetadataApplicationConfig config,
|
||||
Authorizer authorizer,
|
||||
AuthenticatorHandler authHandler)
|
||||
AuthenticatorHandler authHandler,
|
||||
Limits limits)
|
||||
throws ClassNotFoundException,
|
||||
NoSuchMethodException,
|
||||
IllegalAccessException,
|
||||
@ -253,19 +251,36 @@ public final class CollectionRegistry {
|
||||
|
||||
// Create the resource identified by resourceClass
|
||||
try {
|
||||
resource = clz.getDeclaredConstructor(Authorizer.class).newInstance(authorizer);
|
||||
resource =
|
||||
clz.getDeclaredConstructor(OpenMetadataApplicationConfig.class, Limits.class)
|
||||
.newInstance(config, limits);
|
||||
} catch (NoSuchMethodException e) {
|
||||
try {
|
||||
resource =
|
||||
clz.getDeclaredConstructor(Authorizer.class, AuthenticatorHandler.class)
|
||||
.newInstance(authorizer, authHandler);
|
||||
clz.getDeclaredConstructor(Authorizer.class, Limits.class)
|
||||
.newInstance(authorizer, limits);
|
||||
} catch (NoSuchMethodException ex) {
|
||||
try {
|
||||
resource =
|
||||
clz.getDeclaredConstructor(Jdbi.class, Authorizer.class)
|
||||
.newInstance(jdbi, authorizer);
|
||||
resource = clz.getDeclaredConstructor(Authorizer.class).newInstance(authorizer);
|
||||
} catch (NoSuchMethodException exe) {
|
||||
resource = Class.forName(resourceClass).getConstructor().newInstance();
|
||||
try {
|
||||
resource =
|
||||
clz.getDeclaredConstructor(
|
||||
Authorizer.class, Limits.class, AuthenticatorHandler.class)
|
||||
.newInstance(authorizer, limits, authHandler);
|
||||
} catch (NoSuchMethodException exec) {
|
||||
try {
|
||||
resource =
|
||||
clz.getDeclaredConstructor(Jdbi.class, Authorizer.class)
|
||||
.newInstance(jdbi, authorizer);
|
||||
} catch (NoSuchMethodException execp) {
|
||||
try {
|
||||
resource = clz.getDeclaredConstructor(Limits.class).newInstance(limits);
|
||||
} catch (NoSuchMethodException except) {
|
||||
resource = Class.forName(resourceClass).getConstructor().newInstance();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (Exception ex) {
|
||||
|
||||
@ -31,6 +31,7 @@ import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.exception.CatalogExceptionMessage;
|
||||
import org.openmetadata.service.jdbi3.EntityRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.search.SearchListFilter;
|
||||
import org.openmetadata.service.search.SearchSortFilter;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -53,14 +54,16 @@ public abstract class EntityResource<T extends EntityInterface, K extends Entity
|
||||
protected final Set<String> allowedFields;
|
||||
@Getter protected final K repository;
|
||||
protected final Authorizer authorizer;
|
||||
protected final Limits limits;
|
||||
protected final Map<String, MetadataOperation> fieldsToViewOperations = new HashMap<>();
|
||||
|
||||
protected EntityResource(String entityType, Authorizer authorizer) {
|
||||
protected EntityResource(String entityType, Authorizer authorizer, Limits limits) {
|
||||
this.entityType = entityType;
|
||||
this.repository = (K) Entity.getEntityRepository(entityType);
|
||||
this.entityClass = (Class<T>) Entity.getEntityClassFromType(entityType);
|
||||
allowedFields = repository.getAllowedFields();
|
||||
this.authorizer = authorizer;
|
||||
this.limits = limits;
|
||||
addViewOperation(
|
||||
"owner,followers,votes,tags,extension,domain,dataProducts,experts", VIEW_BASIC);
|
||||
Entity.registerResourcePermissions(entityType, getEntitySpecificOperations());
|
||||
@ -259,6 +262,7 @@ public abstract class EntityResource<T extends EntityInterface, K extends Entity
|
||||
OperationContext operationContext = new OperationContext(entityType, CREATE);
|
||||
CreateResourceContext<T> createResourceContext =
|
||||
new CreateResourceContext<>(entityType, entity);
|
||||
limits.enforceLimits(securityContext, createResourceContext, operationContext);
|
||||
authorizer.authorize(securityContext, operationContext, createResourceContext);
|
||||
entity = addHref(uriInfo, repository.create(uriInfo, entity));
|
||||
return Response.created(entity.getHref()).entity(entity).build();
|
||||
@ -274,6 +278,7 @@ public abstract class EntityResource<T extends EntityInterface, K extends Entity
|
||||
if (operation == CREATE) {
|
||||
CreateResourceContext<T> createResourceContext =
|
||||
new CreateResourceContext<>(entityType, entity);
|
||||
limits.enforceLimits(securityContext, createResourceContext, operationContext);
|
||||
authorizer.authorize(securityContext, operationContext, createResourceContext);
|
||||
entity = addHref(uriInfo, repository.create(uriInfo, entity));
|
||||
return new PutResponse<>(Response.Status.CREATED, entity, ENTITY_CREATED).toResponse();
|
||||
@ -315,6 +320,9 @@ public abstract class EntityResource<T extends EntityInterface, K extends Entity
|
||||
DeleteResponse<T> response =
|
||||
repository.delete(securityContext.getUserPrincipal().getName(), id, recursive, hardDelete);
|
||||
repository.deleteFromSearch(response.entity(), response.changeType());
|
||||
if (hardDelete) {
|
||||
limits.invalidateCache(entityType);
|
||||
}
|
||||
addHref(uriInfo, response.entity());
|
||||
return response.toResponse();
|
||||
}
|
||||
|
||||
@ -49,6 +49,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.WebAnalyticEventRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -70,8 +71,8 @@ public class WebAnalyticEventResource
|
||||
public static final String COLLECTION_PATH = WebAnalyticEventRepository.COLLECTION_PATH;
|
||||
static final String FIELDS = "owner";
|
||||
|
||||
public WebAnalyticEventResource(Authorizer authorizer) {
|
||||
super(Entity.WEB_ANALYTIC_EVENT, authorizer);
|
||||
public WebAnalyticEventResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.WEB_ANALYTIC_EVENT, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class WebAnalyticEventList extends ResultList<WebAnalyticEvent> {
|
||||
|
||||
@ -51,6 +51,7 @@ import org.openmetadata.service.apps.ApplicationHandler;
|
||||
import org.openmetadata.service.clients.pipeline.PipelineServiceClientFactory;
|
||||
import org.openmetadata.service.jdbi3.AppMarketPlaceRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -96,8 +97,8 @@ public class AppMarketPlaceResource
|
||||
}
|
||||
}
|
||||
|
||||
public AppMarketPlaceResource(Authorizer authorizer) {
|
||||
super(Entity.APP_MARKET_PLACE_DEF, authorizer);
|
||||
public AppMarketPlaceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.APP_MARKET_PLACE_DEF, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class AppMarketPlaceDefinitionList extends ResultList<AppMarketPlaceDefinition> {
|
||||
|
||||
@ -72,6 +72,7 @@ import org.openmetadata.service.jdbi3.AppRepository;
|
||||
import org.openmetadata.service.jdbi3.CollectionDAO;
|
||||
import org.openmetadata.service.jdbi3.IngestionPipelineRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.search.SearchRepository;
|
||||
@ -162,8 +163,8 @@ public class AppResource extends EntityResource<App, AppRepository> {
|
||||
}
|
||||
}
|
||||
|
||||
public AppResource(Authorizer authorizer) {
|
||||
super(Entity.APPLICATION, authorizer);
|
||||
public AppResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.APPLICATION, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class AppList extends ResultList<App> {
|
||||
@ -545,6 +546,7 @@ public class AppResource extends EntityResource<App, AppRepository> {
|
||||
})
|
||||
public Response create(
|
||||
@Context UriInfo uriInfo, @Context SecurityContext securityContext, @Valid CreateApp create) {
|
||||
|
||||
AppMarketPlaceDefinition definition =
|
||||
repository
|
||||
.getMarketPlace()
|
||||
@ -553,6 +555,10 @@ public class AppResource extends EntityResource<App, AppRepository> {
|
||||
create.getName(),
|
||||
new EntityUtil.Fields(repository.getMarketPlace().getAllowedFields()));
|
||||
App app = getApplication(definition, create, securityContext.getUserPrincipal().getName());
|
||||
limits.enforceLimits(
|
||||
securityContext,
|
||||
getResourceContext(),
|
||||
new OperationContext(Entity.APPLICATION, MetadataOperation.CREATE));
|
||||
if (app.getScheduleType().equals(ScheduleType.Scheduled)) {
|
||||
ApplicationHandler.getInstance()
|
||||
.installApplication(app, Entity.getCollectionDAO(), searchRepository);
|
||||
@ -714,6 +720,7 @@ public class AppResource extends EntityResource<App, AppRepository> {
|
||||
throw new IllegalArgumentException(
|
||||
CatalogExceptionMessage.systemEntityDeleteNotAllowed(app.getName(), "SystemApp"));
|
||||
}
|
||||
limits.invalidateCache(entityType);
|
||||
// Remove from Pipeline Service
|
||||
deleteApp(securityContext, app, hardDelete);
|
||||
return deleteByName(uriInfo, securityContext, name, true, hardDelete);
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.service.clients.pipeline.PipelineServiceClientFactory;
|
||||
import org.openmetadata.service.jdbi3.EntityRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.WorkflowRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.secrets.SecretsManager;
|
||||
@ -86,8 +87,8 @@ public class WorkflowResource extends EntityResource<Workflow, WorkflowRepositor
|
||||
private PipelineServiceClient pipelineServiceClient;
|
||||
private OpenMetadataApplicationConfig openMetadataApplicationConfig;
|
||||
|
||||
public WorkflowResource(Authorizer authorizer) {
|
||||
super(Entity.WORKFLOW, authorizer);
|
||||
public WorkflowResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.WORKFLOW, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -65,6 +65,7 @@ import org.openmetadata.service.jdbi3.CollectionDAO.EntityRelationshipRecord;
|
||||
import org.openmetadata.service.jdbi3.EntityRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.UserRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.resources.teams.RoleResource;
|
||||
@ -87,8 +88,8 @@ import org.openmetadata.service.util.UserUtil;
|
||||
public class BotResource extends EntityResource<Bot, BotRepository> {
|
||||
public static final String COLLECTION_PATH = "/v1/bots/";
|
||||
|
||||
public BotResource(Authorizer authorizer) {
|
||||
super(Entity.BOT, authorizer);
|
||||
public BotResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.BOT, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ChartRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -82,8 +83,8 @@ public class ChartResource extends EntityResource<Chart, ChartRepository> {
|
||||
return chart;
|
||||
}
|
||||
|
||||
public ChartResource(Authorizer authorizer) {
|
||||
super(Entity.CHART, authorizer);
|
||||
public ChartResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.CHART, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DashboardRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -85,8 +86,8 @@ public class DashboardResource extends EntityResource<Dashboard, DashboardReposi
|
||||
return dashboard;
|
||||
}
|
||||
|
||||
public DashboardResource(Authorizer authorizer) {
|
||||
super(Entity.DASHBOARD, authorizer);
|
||||
public DashboardResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DASHBOARD, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -61,6 +61,7 @@ import org.openmetadata.schema.type.csv.CsvImportResult;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DatabaseRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -95,8 +96,8 @@ public class DatabaseResource extends EntityResource<Database, DatabaseRepositor
|
||||
return listOf(MetadataOperation.VIEW_USAGE, MetadataOperation.EDIT_USAGE);
|
||||
}
|
||||
|
||||
public DatabaseResource(Authorizer authorizer) {
|
||||
super(Entity.DATABASE, authorizer);
|
||||
public DatabaseResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DATABASE, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class DatabaseList extends ResultList<Database> {
|
||||
|
||||
@ -56,6 +56,7 @@ import org.openmetadata.schema.type.csv.CsvImportResult;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DatabaseSchemaRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -84,8 +85,8 @@ public class DatabaseSchemaResource
|
||||
return schema;
|
||||
}
|
||||
|
||||
public DatabaseSchemaResource(Authorizer authorizer) {
|
||||
super(Entity.DATABASE_SCHEMA, authorizer);
|
||||
public DatabaseSchemaResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DATABASE_SCHEMA, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -27,6 +27,7 @@ import org.openmetadata.schema.type.Include;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.StoredProcedureRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -54,8 +55,8 @@ public class StoredProcedureResource
|
||||
return storedProcedure;
|
||||
}
|
||||
|
||||
public StoredProcedureResource(Authorizer authorizer) {
|
||||
super(Entity.STORED_PROCEDURE, authorizer);
|
||||
public StoredProcedureResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.STORED_PROCEDURE, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class StoredProcedureList extends ResultList<StoredProcedure> {
|
||||
|
||||
@ -70,6 +70,7 @@ import org.openmetadata.schema.type.csv.CsvImportResult;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TableRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -102,8 +103,8 @@ public class TableResource extends EntityResource<Table, TableRepository> {
|
||||
return table;
|
||||
}
|
||||
|
||||
public TableResource(Authorizer authorizer) {
|
||||
super(Entity.TABLE, authorizer);
|
||||
public TableResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TABLE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -48,6 +48,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.DataInsightChartRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.search.SearchRepository;
|
||||
@ -69,8 +70,8 @@ public class DataInsightChartResource
|
||||
public static final String FIELDS = "owner";
|
||||
private final SearchRepository searchRepository;
|
||||
|
||||
public DataInsightChartResource(Authorizer authorizer) {
|
||||
super(Entity.DATA_INSIGHT_CHART, authorizer);
|
||||
public DataInsightChartResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DATA_INSIGHT_CHART, authorizer, limits);
|
||||
searchRepository = Entity.getSearchRepository();
|
||||
}
|
||||
|
||||
|
||||
@ -53,6 +53,7 @@ import org.openmetadata.schema.type.Include;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DashboardDataModelRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.resources.databases.DatabaseUtil;
|
||||
@ -80,8 +81,8 @@ public class DashboardDataModelResource
|
||||
return dashboardDataModel;
|
||||
}
|
||||
|
||||
public DashboardDataModelResource(Authorizer authorizer) {
|
||||
super(Entity.DASHBOARD_DATA_MODEL, authorizer);
|
||||
public DashboardDataModelResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DASHBOARD_DATA_MODEL, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class DashboardDataModelList extends ResultList<DashboardDataModel> {
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.DocumentRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -83,8 +84,8 @@ public class DocStoreResource extends EntityResource<Document, DocumentRepositor
|
||||
return listOf(MetadataOperation.EDIT_ALL);
|
||||
}
|
||||
|
||||
public DocStoreResource(Authorizer authorizer) {
|
||||
super(Entity.DOCUMENT, authorizer);
|
||||
public DocStoreResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DOCUMENT, authorizer, limits);
|
||||
}
|
||||
|
||||
public static class DocumentList extends ResultList<Document> {
|
||||
|
||||
@ -60,6 +60,7 @@ import org.openmetadata.schema.type.api.BulkOperationResult;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DataProductRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -80,8 +81,8 @@ public class DataProductResource extends EntityResource<DataProduct, DataProduct
|
||||
public static final String COLLECTION_PATH = "/v1/dataProducts/";
|
||||
static final String FIELDS = "domain,owner,experts,assets";
|
||||
|
||||
public DataProductResource(Authorizer authorizer) {
|
||||
super(Entity.DATA_PRODUCT, authorizer);
|
||||
public DataProductResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DATA_PRODUCT, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -55,6 +55,7 @@ import org.openmetadata.schema.type.api.BulkOperationResult;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DomainRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -74,8 +75,8 @@ public class DomainResource extends EntityResource<Domain, DomainRepository> {
|
||||
public static final String COLLECTION_PATH = "/v1/domains/";
|
||||
static final String FIELDS = "children,owner,experts";
|
||||
|
||||
public DomainResource(Authorizer authorizer) {
|
||||
super(Entity.DOMAIN, authorizer);
|
||||
public DomainResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DOMAIN, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -56,6 +56,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.Filter;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TestCaseRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.resources.feeds.MessageParser.EntityLink;
|
||||
@ -63,6 +64,7 @@ import org.openmetadata.service.search.SearchListFilter;
|
||||
import org.openmetadata.service.search.SearchSortFilter;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
import org.openmetadata.service.security.mask.PIIMasker;
|
||||
import org.openmetadata.service.security.policyevaluator.CreateResourceContext;
|
||||
import org.openmetadata.service.security.policyevaluator.OperationContext;
|
||||
import org.openmetadata.service.security.policyevaluator.ResourceContext;
|
||||
import org.openmetadata.service.security.policyevaluator.ResourceContextInterface;
|
||||
@ -99,8 +101,8 @@ public class TestCaseResource extends EntityResource<TestCase, TestCaseRepositor
|
||||
return test;
|
||||
}
|
||||
|
||||
public TestCaseResource(Authorizer authorizer) {
|
||||
super(Entity.TEST_CASE, authorizer);
|
||||
public TestCaseResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TEST_CASE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
@ -630,6 +632,10 @@ public class TestCaseResource extends EntityResource<TestCase, TestCaseRepositor
|
||||
new OperationContext(Entity.TABLE, MetadataOperation.EDIT_TESTS);
|
||||
ResourceContextInterface resourceContext =
|
||||
TestCaseResourceContext.builder().entityLink(entityLink).build();
|
||||
limits.enforceLimits(
|
||||
securityContext,
|
||||
new CreateResourceContext<>(entityType, test),
|
||||
new OperationContext(Entity.TEST_CASE, MetadataOperation.EDIT_TESTS));
|
||||
authorizer.authorize(securityContext, operationContext, resourceContext);
|
||||
repository.isTestSuiteExecutable(create.getTestSuite());
|
||||
test = addHref(uriInfo, repository.create(uriInfo, test));
|
||||
|
||||
@ -45,6 +45,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TestDefinitionRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -65,8 +66,8 @@ public class TestDefinitionResource
|
||||
public static final String COLLECTION_PATH = "/v1/dataQuality/testDefinitions";
|
||||
static final String FIELDS = "owner";
|
||||
|
||||
public TestDefinitionResource(Authorizer authorizer) {
|
||||
super(Entity.TEST_DEFINITION, authorizer);
|
||||
public TestDefinitionResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TEST_DEFINITION, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -49,6 +49,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TestSuiteRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.search.SearchListFilter;
|
||||
@ -78,8 +79,8 @@ public class TestSuiteResource extends EntityResource<TestSuite, TestSuiteReposi
|
||||
static final String FIELDS = "owner,tests,summary";
|
||||
static final String SEARCH_FIELDS_EXCLUDE = "table,database,databaseSchema,service";
|
||||
|
||||
public TestSuiteResource(Authorizer authorizer) {
|
||||
super(Entity.TEST_SUITE, authorizer);
|
||||
public TestSuiteResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TEST_SUITE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -74,6 +74,7 @@ import org.openmetadata.service.events.subscription.EventsSubscriptionRegistry;
|
||||
import org.openmetadata.service.jdbi3.CollectionDAO;
|
||||
import org.openmetadata.service.jdbi3.EventSubscriptionRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -96,8 +97,8 @@ public class EventSubscriptionResource
|
||||
public static final String COLLECTION_PATH = "/v1/events/subscriptions";
|
||||
public static final String FIELDS = "owner,filteringRules";
|
||||
|
||||
public EventSubscriptionResource(Authorizer authorizer) {
|
||||
super(Entity.EVENT_SUBSCRIPTION, authorizer);
|
||||
public EventSubscriptionResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.EVENT_SUBSCRIPTION, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -58,6 +58,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.GlossaryRepository;
|
||||
import org.openmetadata.service.jdbi3.GlossaryRepository.GlossaryCsv;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -77,8 +78,8 @@ public class GlossaryResource extends EntityResource<Glossary, GlossaryRepositor
|
||||
public static final String COLLECTION_PATH = "v1/glossaries/";
|
||||
static final String FIELDS = "owner,tags,reviewers,usageCount,termCount,domain,extension";
|
||||
|
||||
public GlossaryResource(Authorizer authorizer) {
|
||||
super(Entity.GLOSSARY, authorizer);
|
||||
public GlossaryResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.GLOSSARY, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -69,6 +69,7 @@ import org.openmetadata.service.jdbi3.EntityRepository;
|
||||
import org.openmetadata.service.jdbi3.GlossaryRepository;
|
||||
import org.openmetadata.service.jdbi3.GlossaryTermRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -100,8 +101,8 @@ public class GlossaryTermResource extends EntityResource<GlossaryTerm, GlossaryT
|
||||
return term;
|
||||
}
|
||||
|
||||
public GlossaryTermResource(Authorizer authorizer) {
|
||||
super(Entity.GLOSSARY_TERM, authorizer);
|
||||
public GlossaryTermResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.GLOSSARY_TERM, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -45,6 +45,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.EntityTimeSeriesDAO.OrderBy;
|
||||
import org.openmetadata.service.jdbi3.KpiRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -68,8 +69,8 @@ public class KpiResource extends EntityResource<Kpi, KpiRepository> {
|
||||
return kpi;
|
||||
}
|
||||
|
||||
public KpiResource(Authorizer authorizer) {
|
||||
super(Entity.KPI, authorizer);
|
||||
public KpiResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.KPI, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -0,0 +1,84 @@
|
||||
package org.openmetadata.service.resources.limits;
|
||||
|
||||
import io.swagger.v3.oas.annotations.Hidden;
|
||||
import io.swagger.v3.oas.annotations.Operation;
|
||||
import io.swagger.v3.oas.annotations.Parameter;
|
||||
import io.swagger.v3.oas.annotations.media.Content;
|
||||
import io.swagger.v3.oas.annotations.media.Schema;
|
||||
import io.swagger.v3.oas.annotations.responses.ApiResponse;
|
||||
import io.swagger.v3.oas.annotations.tags.Tag;
|
||||
import javax.ws.rs.DefaultValue;
|
||||
import javax.ws.rs.GET;
|
||||
import javax.ws.rs.Path;
|
||||
import javax.ws.rs.PathParam;
|
||||
import javax.ws.rs.Produces;
|
||||
import javax.ws.rs.QueryParam;
|
||||
import javax.ws.rs.core.Context;
|
||||
import javax.ws.rs.core.MediaType;
|
||||
import javax.ws.rs.core.Response;
|
||||
import javax.ws.rs.core.SecurityContext;
|
||||
import javax.ws.rs.core.UriInfo;
|
||||
import org.openmetadata.schema.system.LimitsConfig;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
|
||||
@Path("/v1/limits")
|
||||
@Tag(name = "Limits", description = "APIs related to Limits configuration and settings.")
|
||||
@Hidden
|
||||
@Produces(MediaType.APPLICATION_JSON)
|
||||
@Collection(name = "limits")
|
||||
public class LimitsResource {
|
||||
private final Limits limits;
|
||||
private final OpenMetadataApplicationConfig config;
|
||||
|
||||
public LimitsResource(OpenMetadataApplicationConfig config, Limits limits) {
|
||||
this.limits = limits;
|
||||
this.config = config;
|
||||
}
|
||||
|
||||
@GET
|
||||
@Path("/features/{name}")
|
||||
@Operation(
|
||||
operationId = "getLimitsForaFeature",
|
||||
summary = "Get Limits configuration for a feature",
|
||||
responses = {
|
||||
@ApiResponse(responseCode = "200", description = "Limits configuration for a feature")
|
||||
})
|
||||
public Response getLimitsForaFeature(
|
||||
@Context UriInfo uriInfo,
|
||||
@Context SecurityContext securityContext,
|
||||
@Parameter(description = "Name of the Feature", schema = @Schema(type = "string"))
|
||||
@PathParam("name")
|
||||
String name,
|
||||
@Parameter(
|
||||
description = "Use Cache to retrieve the values.",
|
||||
schema = @Schema(type = "boolean", example = "true"))
|
||||
@QueryParam("cache")
|
||||
@DefaultValue("true")
|
||||
boolean cache) {
|
||||
return limits.getLimitsForaFeature(name, cache);
|
||||
}
|
||||
|
||||
@GET
|
||||
@Path(("/config"))
|
||||
@Operation(
|
||||
operationId = "getLimitsConfiguration",
|
||||
summary = "Get Limits configuration",
|
||||
responses = {
|
||||
@ApiResponse(
|
||||
responseCode = "200",
|
||||
description = "Limits configuration",
|
||||
content =
|
||||
@Content(
|
||||
mediaType = "application/json",
|
||||
schema = @Schema(implementation = LimitsConfig.class)))
|
||||
})
|
||||
public LimitsConfig getAuthConfig() {
|
||||
LimitsConfig limitsConfig = new LimitsConfig();
|
||||
if (limits != null) {
|
||||
limitsConfig = limits.getLimitsConfig();
|
||||
}
|
||||
return limitsConfig;
|
||||
}
|
||||
}
|
||||
@ -48,6 +48,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.MetricsRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -66,8 +67,8 @@ public class MetricsResource extends EntityResource<Metrics, MetricsRepository>
|
||||
public static final String COLLECTION_PATH = "/v1/metrics/";
|
||||
static final String FIELDS = "owner,usageSummary,domain";
|
||||
|
||||
public MetricsResource(Authorizer authorizer) {
|
||||
super(Entity.METRICS, authorizer);
|
||||
public MetricsResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.METRICS, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.MlModelRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -83,8 +84,8 @@ public class MlModelResource extends EntityResource<MlModel, MlModelRepository>
|
||||
return mlmodel;
|
||||
}
|
||||
|
||||
public MlModelResource(Authorizer authorizer) {
|
||||
super(Entity.MLMODEL, authorizer);
|
||||
public MlModelResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.MLMODEL, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -59,6 +59,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.PipelineRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.resources.dqtests.TestCaseResource;
|
||||
@ -86,8 +87,8 @@ public class PipelineResource extends EntityResource<Pipeline, PipelineRepositor
|
||||
return pipeline;
|
||||
}
|
||||
|
||||
public PipelineResource(Authorizer authorizer) {
|
||||
super(Entity.PIPELINE, authorizer);
|
||||
public PipelineResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.PIPELINE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -62,6 +62,7 @@ import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.ResourceRegistry;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.PolicyRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.CollectionRegistry;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
@ -92,8 +93,8 @@ public class PolicyResource extends EntityResource<Policy, PolicyRepository> {
|
||||
return policy;
|
||||
}
|
||||
|
||||
public PolicyResource(Authorizer authorizer) {
|
||||
super(Entity.POLICY, authorizer);
|
||||
public PolicyResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.POLICY, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -46,6 +46,7 @@ import org.openmetadata.schema.type.Votes;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.QueryRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -66,8 +67,8 @@ public class QueryResource extends EntityResource<Query, QueryRepository> {
|
||||
public static final String COLLECTION_PATH = "v1/queries/";
|
||||
static final String FIELDS = "owner,followers,users,votes,tags,queryUsedIn";
|
||||
|
||||
public QueryResource(Authorizer authorizer) {
|
||||
super(Entity.QUERY, authorizer);
|
||||
public QueryResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.QUERY, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -46,6 +46,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.ReportRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -65,8 +66,8 @@ public class ReportResource extends EntityResource<Report, ReportRepository> {
|
||||
public static final String COLLECTION_PATH = "/v1/reports/";
|
||||
static final String FIELDS = "owner,usageSummary";
|
||||
|
||||
public ReportResource(Authorizer authorizer) {
|
||||
super(Entity.REPORT, authorizer);
|
||||
public ReportResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.REPORT, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -58,6 +58,7 @@ import org.openmetadata.schema.type.searchindex.SearchIndexSampleData;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.SearchIndexRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -84,8 +85,8 @@ public class SearchIndexResource extends EntityResource<SearchIndex, SearchIndex
|
||||
return searchIndex;
|
||||
}
|
||||
|
||||
public SearchIndexResource(Authorizer authorizer) {
|
||||
super(Entity.SEARCH_INDEX, authorizer);
|
||||
public SearchIndexResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.SEARCH_INDEX, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -28,6 +28,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.exception.InvalidServiceConnectionException;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.ServiceEntityRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.secrets.SecretsManager;
|
||||
import org.openmetadata.service.secrets.SecretsManagerFactory;
|
||||
@ -47,8 +48,8 @@ public abstract class ServiceEntityResource<
|
||||
private final ServiceType serviceType;
|
||||
|
||||
protected ServiceEntityResource(
|
||||
String entityType, Authorizer authorizer, ServiceType serviceType) {
|
||||
super(entityType, authorizer);
|
||||
String entityType, Authorizer authorizer, Limits limits, ServiceType serviceType) {
|
||||
super(entityType, authorizer, limits);
|
||||
this.serviceType = serviceType;
|
||||
serviceEntityRepository =
|
||||
(ServiceEntityRepository<T, S>) Entity.getServiceEntityRepository(serviceType);
|
||||
|
||||
@ -33,6 +33,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TestConnectionDefinitionRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -53,8 +54,8 @@ public class TestConnectionDefinitionResource
|
||||
public static final String COLLECTION_PATH = "/v1/services/testConnectionDefinitions";
|
||||
static final String FIELDS = "owner";
|
||||
|
||||
public TestConnectionDefinitionResource(Authorizer authorizer) {
|
||||
super(Entity.TEST_CONNECTION_DEFINITION, authorizer);
|
||||
public TestConnectionDefinitionResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TEST_CONNECTION_DEFINITION, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.schema.type.Include;
|
||||
import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DashboardServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -75,8 +76,8 @@ public class DashboardServiceResource
|
||||
public static final String COLLECTION_PATH = "v1/services/dashboardServices";
|
||||
static final String FIELDS = "owner,domain";
|
||||
|
||||
public DashboardServiceResource(Authorizer authorizer) {
|
||||
super(Entity.DASHBOARD_SERVICE, authorizer, ServiceType.DASHBOARD);
|
||||
public DashboardServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DASHBOARD_SERVICE, authorizer, limits, ServiceType.DASHBOARD);
|
||||
}
|
||||
|
||||
public static class DashboardServiceList extends ResultList<DashboardService> {
|
||||
|
||||
@ -59,6 +59,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.schema.type.csv.CsvImportResult;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.DatabaseServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -94,8 +95,8 @@ public class DatabaseServiceResource
|
||||
return null;
|
||||
}
|
||||
|
||||
public DatabaseServiceResource(Authorizer authorizer) {
|
||||
super(Entity.DATABASE_SERVICE, authorizer, ServiceType.DATABASE);
|
||||
public DatabaseServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.DATABASE_SERVICE, authorizer, limits, ServiceType.DATABASE);
|
||||
}
|
||||
|
||||
public static class DatabaseServiceList extends ResultList<DatabaseService> {
|
||||
|
||||
@ -71,6 +71,7 @@ import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.clients.pipeline.PipelineServiceClientFactory;
|
||||
import org.openmetadata.service.jdbi3.IngestionPipelineRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.secrets.SecretsManager;
|
||||
@ -78,6 +79,7 @@ import org.openmetadata.service.secrets.SecretsManagerFactory;
|
||||
import org.openmetadata.service.secrets.masker.EntityMaskerFactory;
|
||||
import org.openmetadata.service.security.AuthorizationException;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
import org.openmetadata.service.security.policyevaluator.CreateResourceContext;
|
||||
import org.openmetadata.service.security.policyevaluator.OperationContext;
|
||||
import org.openmetadata.service.util.EntityUtil.Fields;
|
||||
import org.openmetadata.service.util.OpenMetadataConnectionBuilder;
|
||||
@ -107,8 +109,8 @@ public class IngestionPipelineResource
|
||||
return ingestionPipeline;
|
||||
}
|
||||
|
||||
public IngestionPipelineResource(Authorizer authorizer) {
|
||||
super(Entity.INGESTION_PIPELINE, authorizer);
|
||||
public IngestionPipelineResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.INGESTION_PIPELINE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
@ -489,7 +491,7 @@ public class IngestionPipelineResource
|
||||
@Path("/deploy/{id}")
|
||||
@Operation(
|
||||
summary = "Deploy an ingestion pipeline run",
|
||||
description = "Trigger a ingestion pipeline run by Id.",
|
||||
description = "Deploy a ingestion pipeline run by Id.",
|
||||
responses = {
|
||||
@ApiResponse(
|
||||
responseCode = "200",
|
||||
@ -922,6 +924,10 @@ public class IngestionPipelineResource
|
||||
UUID id, UriInfo uriInfo, SecurityContext securityContext) {
|
||||
Fields fields = getFields(FIELD_OWNER);
|
||||
IngestionPipeline ingestionPipeline = repository.get(uriInfo, id, fields);
|
||||
CreateResourceContext<IngestionPipeline> createResourceContext =
|
||||
new CreateResourceContext<>(entityType, ingestionPipeline);
|
||||
OperationContext operationContext = new OperationContext(entityType, MetadataOperation.DEPLOY);
|
||||
limits.enforceLimits(securityContext, createResourceContext, operationContext);
|
||||
decryptOrNullify(securityContext, ingestionPipeline, true);
|
||||
ServiceEntityInterface service =
|
||||
Entity.getEntity(ingestionPipeline.getService(), "", Include.NON_DELETED);
|
||||
@ -937,6 +943,10 @@ public class IngestionPipelineResource
|
||||
UUID id, UriInfo uriInfo, SecurityContext securityContext, String botName) {
|
||||
Fields fields = getFields(FIELD_OWNER);
|
||||
IngestionPipeline ingestionPipeline = repository.get(uriInfo, id, fields);
|
||||
CreateResourceContext<IngestionPipeline> createResourceContext =
|
||||
new CreateResourceContext<>(entityType, ingestionPipeline);
|
||||
OperationContext operationContext = new OperationContext(entityType, MetadataOperation.TRIGGER);
|
||||
limits.enforceLimits(securityContext, createResourceContext, operationContext);
|
||||
if (CommonUtil.nullOrEmpty(botName)) {
|
||||
// Use Default Ingestion Bot
|
||||
ingestionPipeline.setOpenMetadataServerConnection(
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.schema.type.MessagingConnection;
|
||||
import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.MessagingServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -75,8 +76,8 @@ public class MessagingServiceResource
|
||||
public static final String COLLECTION_PATH = "v1/services/messagingServices/";
|
||||
public static final String FIELDS = "owner,domain";
|
||||
|
||||
public MessagingServiceResource(Authorizer authorizer) {
|
||||
super(Entity.MESSAGING_SERVICE, authorizer, ServiceType.MESSAGING);
|
||||
public MessagingServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.MESSAGING_SERVICE, authorizer, limits, ServiceType.MESSAGING);
|
||||
}
|
||||
|
||||
public static class MessagingServiceList extends ResultList<MessagingService> {
|
||||
|
||||
@ -56,6 +56,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.MetadataServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -119,8 +120,8 @@ public class MetadataServiceResource
|
||||
return service;
|
||||
}
|
||||
|
||||
public MetadataServiceResource(Authorizer authorizer) {
|
||||
super(Entity.METADATA_SERVICE, authorizer, ServiceType.METADATA);
|
||||
public MetadataServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.METADATA_SERVICE, authorizer, limits, ServiceType.METADATA);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -59,6 +59,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.schema.type.MlModelConnection;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.MlModelServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -83,8 +84,8 @@ public class MlModelServiceResource
|
||||
return service;
|
||||
}
|
||||
|
||||
public MlModelServiceResource(Authorizer authorizer) {
|
||||
super(Entity.MLMODEL_SERVICE, authorizer, ServiceType.ML_MODEL);
|
||||
public MlModelServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.MLMODEL_SERVICE, authorizer, limits, ServiceType.ML_MODEL);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -57,6 +57,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.schema.type.PipelineConnection;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.PipelineServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -81,8 +82,8 @@ public class PipelineServiceResource
|
||||
return service;
|
||||
}
|
||||
|
||||
public PipelineServiceResource(Authorizer authorizer) {
|
||||
super(Entity.PIPELINE_SERVICE, authorizer, ServiceType.PIPELINE);
|
||||
public PipelineServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.PIPELINE_SERVICE, authorizer, limits, ServiceType.PIPELINE);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -45,6 +45,7 @@ import org.openmetadata.schema.type.SearchConnection;
|
||||
import org.openmetadata.schema.utils.EntityInterfaceUtil;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.SearchServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -72,8 +73,8 @@ public class SearchServiceResource
|
||||
return service;
|
||||
}
|
||||
|
||||
public SearchServiceResource(Authorizer authorizer) {
|
||||
super(Entity.SEARCH_SERVICE, authorizer, ServiceType.SEARCH);
|
||||
public SearchServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.SEARCH_SERVICE, authorizer, limits, ServiceType.SEARCH);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -45,6 +45,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.schema.type.StorageConnection;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.StorageServiceRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.services.ServiceEntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -72,8 +73,8 @@ public class StorageServiceResource
|
||||
return service;
|
||||
}
|
||||
|
||||
public StorageServiceResource(Authorizer authorizer) {
|
||||
super(Entity.STORAGE_SERVICE, authorizer, ServiceType.STORAGE);
|
||||
public StorageServiceResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.STORAGE_SERVICE, authorizer, limits, ServiceType.STORAGE);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -42,6 +42,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ContainerRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -70,8 +71,8 @@ public class ContainerResource extends EntityResource<Container, ContainerReposi
|
||||
return container;
|
||||
}
|
||||
|
||||
public ContainerResource(Authorizer authorizer) {
|
||||
super(Entity.CONTAINER, authorizer);
|
||||
public ContainerResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.CONTAINER, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -55,6 +55,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ClassificationRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -84,8 +85,8 @@ public class ClassificationResource
|
||||
/* Required for serde */
|
||||
}
|
||||
|
||||
public ClassificationResource(Authorizer authorizer) {
|
||||
super(Entity.CLASSIFICATION, authorizer);
|
||||
public ClassificationResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.CLASSIFICATION, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -64,6 +64,7 @@ import org.openmetadata.service.jdbi3.ClassificationRepository;
|
||||
import org.openmetadata.service.jdbi3.EntityRepository;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TagRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -94,8 +95,8 @@ public class TagResource extends EntityResource<Tag, TagRepository> {
|
||||
/* Required for serde */
|
||||
}
|
||||
|
||||
public TagResource(Authorizer authorizer) {
|
||||
super(Entity.TAG, authorizer);
|
||||
public TagResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TAG, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -42,6 +42,7 @@ import org.openmetadata.schema.type.MetadataOperation;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.PersonaRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -69,8 +70,8 @@ public class PersonaResource extends EntityResource<Persona, PersonaRepository>
|
||||
return persona;
|
||||
}
|
||||
|
||||
public PersonaResource(Authorizer authorizer) {
|
||||
super(Entity.PERSONA, authorizer);
|
||||
public PersonaResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.PERSONA, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -59,6 +59,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.RoleRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -92,8 +93,8 @@ public class RoleResource extends EntityResource<Role, RoleRepository> {
|
||||
return role;
|
||||
}
|
||||
|
||||
public RoleResource(Authorizer authorizer) {
|
||||
super(Entity.ROLE, authorizer);
|
||||
public RoleResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.ROLE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -67,6 +67,7 @@ import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TeamRepository;
|
||||
import org.openmetadata.service.jdbi3.TeamRepository.TeamCsv;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -103,8 +104,8 @@ public class TeamResource extends EntityResource<Team, TeamRepository> {
|
||||
return team;
|
||||
}
|
||||
|
||||
public TeamResource(Authorizer authorizer) {
|
||||
super(Entity.TEAM, authorizer);
|
||||
public TeamResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TEAM, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -22,6 +22,7 @@ import static org.openmetadata.schema.api.teams.CreateUser.CreatePasswordType.AD
|
||||
import static org.openmetadata.schema.auth.ChangePasswordRequest.RequestType.SELF;
|
||||
import static org.openmetadata.schema.entity.teams.AuthenticationMechanism.AuthType.BASIC;
|
||||
import static org.openmetadata.schema.entity.teams.AuthenticationMechanism.AuthType.JWT;
|
||||
import static org.openmetadata.schema.type.Include.ALL;
|
||||
import static org.openmetadata.service.exception.CatalogExceptionMessage.EMAIL_SENDING_ISSUE;
|
||||
import static org.openmetadata.service.jdbi3.UserRepository.AUTH_MECHANISM_FIELD;
|
||||
import static org.openmetadata.service.security.jwt.JWTTokenGenerator.getExpiryDate;
|
||||
@ -124,6 +125,7 @@ import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TokenRepository;
|
||||
import org.openmetadata.service.jdbi3.UserRepository;
|
||||
import org.openmetadata.service.jdbi3.UserRepository.UserCsv;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.secrets.SecretsManager;
|
||||
@ -186,8 +188,9 @@ public class UserResource extends EntityResource<User, UserRepository> {
|
||||
return user;
|
||||
}
|
||||
|
||||
public UserResource(Authorizer authorizer, AuthenticatorHandler authenticatorHandler) {
|
||||
super(Entity.USER, authorizer);
|
||||
public UserResource(
|
||||
Authorizer authorizer, Limits limits, AuthenticatorHandler authenticatorHandler) {
|
||||
super(Entity.USER, authorizer, limits);
|
||||
jwtTokenGenerator = JWTTokenGenerator.getInstance();
|
||||
allowedFields.remove(USER_PROTECTED_FIELDS);
|
||||
tokenRepository = Entity.getTokenRepository();
|
||||
@ -676,7 +679,13 @@ public class UserResource extends EntityResource<User, UserRepository> {
|
||||
@Valid CreateUser create) {
|
||||
User user = getUser(securityContext.getUserPrincipal().getName(), create);
|
||||
repository.prepareInternal(user, true);
|
||||
|
||||
User existingUser = repository.findByNameOrNull(user.getFullyQualifiedName(), ALL);
|
||||
if (existingUser == null) {
|
||||
limits.enforceLimits(
|
||||
securityContext,
|
||||
getResourceContextByName(user.getFullyQualifiedName()),
|
||||
new OperationContext(entityType, MetadataOperation.CREATE));
|
||||
}
|
||||
ResourceContext<?> resourceContext = getResourceContextByName(user.getFullyQualifiedName());
|
||||
if (Boolean.TRUE.equals(create.getIsAdmin()) || Boolean.TRUE.equals(create.getIsBot())) {
|
||||
authorizer.authorizeAdmin(securityContext);
|
||||
@ -829,11 +838,14 @@ public class UserResource extends EntityResource<User, UserRepository> {
|
||||
@Context SecurityContext securityContext,
|
||||
@Parameter(description = "Id of the user", schema = @Schema(type = "UUID")) @PathParam("id")
|
||||
UUID id) {
|
||||
|
||||
User user = repository.get(uriInfo, id, new Fields(Set.of(AUTH_MECHANISM_FIELD)));
|
||||
if (!Boolean.TRUE.equals(user.getIsBot())) {
|
||||
throw new IllegalArgumentException("JWT token is only supported for bot users");
|
||||
}
|
||||
limits.enforceLimits(
|
||||
securityContext,
|
||||
getResourceContext(),
|
||||
new OperationContext(entityType, MetadataOperation.GENERATE_TOKEN));
|
||||
decryptOrNullify(securityContext, user);
|
||||
authorizer.authorizeAdmin(securityContext);
|
||||
return user.getAuthenticationMechanism();
|
||||
@ -1249,6 +1261,10 @@ public class UserResource extends EntityResource<User, UserRepository> {
|
||||
@Parameter(description = "User Name of the User for which to get. (Default = `false`)")
|
||||
@QueryParam("username")
|
||||
String userName) {
|
||||
limits.enforceLimits(
|
||||
securityContext,
|
||||
getResourceContext(),
|
||||
new OperationContext(entityType, MetadataOperation.GENERATE_TOKEN));
|
||||
if (userName != null) {
|
||||
authorizer.authorizeAdmin(securityContext);
|
||||
} else {
|
||||
@ -1327,6 +1343,10 @@ public class UserResource extends EntityResource<User, UserRepository> {
|
||||
@Context UriInfo uriInfo,
|
||||
@Context SecurityContext securityContext,
|
||||
@Valid CreatePersonalToken tokenRequest) {
|
||||
limits.enforceLimits(
|
||||
securityContext,
|
||||
getResourceContext(),
|
||||
new OperationContext(entityType, MetadataOperation.GENERATE_TOKEN));
|
||||
String userName = securityContext.getUserPrincipal().getName();
|
||||
User user =
|
||||
repository.getByName(
|
||||
|
||||
@ -58,6 +58,7 @@ import org.openmetadata.schema.type.topic.TopicSampleData;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TopicRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -85,8 +86,8 @@ public class TopicResource extends EntityResource<Topic, TopicRepository> {
|
||||
return topic;
|
||||
}
|
||||
|
||||
public TopicResource(Authorizer authorizer) {
|
||||
super(Entity.TOPIC, authorizer);
|
||||
public TopicResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TOPIC, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -59,6 +59,7 @@ import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.OpenMetadataApplicationConfig;
|
||||
import org.openmetadata.service.jdbi3.ListFilter;
|
||||
import org.openmetadata.service.jdbi3.TypeRepository;
|
||||
import org.openmetadata.service.limits.Limits;
|
||||
import org.openmetadata.service.resources.Collection;
|
||||
import org.openmetadata.service.resources.EntityResource;
|
||||
import org.openmetadata.service.security.Authorizer;
|
||||
@ -88,8 +89,8 @@ public class TypeResource extends EntityResource<Type, TypeRepository> {
|
||||
return type;
|
||||
}
|
||||
|
||||
public TypeResource(Authorizer authorizer) {
|
||||
super(Entity.TYPE, authorizer);
|
||||
public TypeResource(Authorizer authorizer, Limits limits) {
|
||||
super(Entity.TYPE, authorizer, limits);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -8,6 +8,7 @@ import org.openmetadata.schema.tests.TestSuite;
|
||||
import org.openmetadata.schema.type.EntityReference;
|
||||
import org.openmetadata.schema.type.Include;
|
||||
import org.openmetadata.service.Entity;
|
||||
import org.openmetadata.service.search.ParseTags;
|
||||
import org.openmetadata.service.search.SearchIndexUtils;
|
||||
import org.openmetadata.service.search.models.SearchSuggest;
|
||||
|
||||
@ -30,6 +31,8 @@ public record TestSuiteIndex(TestSuite testSuite) implements SearchIndex {
|
||||
doc.put("entityType", Entity.TEST_SUITE);
|
||||
doc.put("owner", getEntityWithDisplayName(testSuite.getOwner()));
|
||||
doc.put("followers", SearchIndexUtils.parseFollowers(testSuite.getFollowers()));
|
||||
ParseTags parseTags = new ParseTags(Entity.getEntityTags(Entity.TEST_SUITE, testSuite));
|
||||
doc.put("tags", parseTags.getTags());
|
||||
setParentRelationships(doc, testSuite);
|
||||
return doc;
|
||||
}
|
||||
|
||||
@ -255,7 +255,7 @@ public class OpenMetadataOperations implements Callable<Integer> {
|
||||
CollectionRegistry.initialize();
|
||||
ApplicationHandler.initialize(config);
|
||||
// load seed data so that repositories are initialized
|
||||
CollectionRegistry.getInstance().loadSeedData(jdbi, config, null, null, true);
|
||||
CollectionRegistry.getInstance().loadSeedData(jdbi, config, null, null, null, true);
|
||||
ApplicationHandler.initialize(config);
|
||||
// creates the default search index application
|
||||
AppScheduler.initialize(config, collectionDAO, searchRepository);
|
||||
|
||||
@ -174,6 +174,73 @@
|
||||
"executable": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"tags": {
|
||||
"properties": {
|
||||
"tagFQN": {
|
||||
"type": "keyword",
|
||||
"normalizer": "lowercase_normalizer"
|
||||
},
|
||||
"labelType": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"description": {
|
||||
"type": "text"
|
||||
},
|
||||
"source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"state": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"domain" : {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 36
|
||||
}
|
||||
}
|
||||
},
|
||||
"type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"name": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"displayName": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"normalizer": "lowercase_normalizer",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"fullyQualifiedName": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"description": {
|
||||
"type": "text"
|
||||
},
|
||||
"deleted": {
|
||||
"type": "text"
|
||||
},
|
||||
"href": {
|
||||
"type": "text"
|
||||
}
|
||||
}
|
||||
},
|
||||
"table": {
|
||||
"properties": {
|
||||
"id": {
|
||||
|
||||
@ -94,14 +94,14 @@
|
||||
"indexName": "test_case_search_index",
|
||||
"indexMappingFile": "/elasticsearch/%s/test_case_index_mapping.json",
|
||||
"alias": "testCase",
|
||||
"parentAliases": ["testSuite"],
|
||||
"parentAliases": ["testSuite", "all"],
|
||||
"childAliases": ["testCaseResolutionStatus"]
|
||||
},
|
||||
"testSuite": {
|
||||
"indexName": "test_suite_search_index",
|
||||
"indexMappingFile": "/elasticsearch/%s/test_suite_index_mapping.json",
|
||||
"alias": "testSuite",
|
||||
"parentAliases": [],
|
||||
"parentAliases": ["all"],
|
||||
"childAliases": ["testCase", "testCaseResolutionStatus"]
|
||||
},
|
||||
"dataProduct": {
|
||||
|
||||
@ -170,6 +170,73 @@
|
||||
"executable": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"tags": {
|
||||
"properties": {
|
||||
"tagFQN": {
|
||||
"type": "keyword",
|
||||
"normalizer": "lowercase_normalizer"
|
||||
},
|
||||
"labelType": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"description": {
|
||||
"type": "text"
|
||||
},
|
||||
"source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"state": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"domain" : {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 36
|
||||
}
|
||||
}
|
||||
},
|
||||
"type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"name": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"displayName": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"normalizer": "lowercase_normalizer",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"fullyQualifiedName": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"description": {
|
||||
"type": "text"
|
||||
},
|
||||
"deleted": {
|
||||
"type": "text"
|
||||
},
|
||||
"href": {
|
||||
"type": "text"
|
||||
}
|
||||
}
|
||||
},
|
||||
"table": {
|
||||
"properties": {
|
||||
"id": {
|
||||
|
||||
@ -159,6 +159,73 @@
|
||||
"executable": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"tags": {
|
||||
"properties": {
|
||||
"tagFQN": {
|
||||
"type": "keyword",
|
||||
"normalizer": "lowercase_normalizer"
|
||||
},
|
||||
"labelType": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"description": {
|
||||
"type": "text"
|
||||
},
|
||||
"source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"state": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"domain" : {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 36
|
||||
}
|
||||
}
|
||||
},
|
||||
"type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"name": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"displayName": {
|
||||
"type": "keyword",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"normalizer": "lowercase_normalizer",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"fullyQualifiedName": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"description": {
|
||||
"type": "text"
|
||||
},
|
||||
"deleted": {
|
||||
"type": "text"
|
||||
},
|
||||
"href": {
|
||||
"type": "text"
|
||||
}
|
||||
}
|
||||
},
|
||||
"table": {
|
||||
"properties": {
|
||||
"id": {
|
||||
|
||||
@ -990,7 +990,7 @@ public class TestSuiteResourceTest extends EntityResourceTest<TestSuite, CreateT
|
||||
? getEntityByName(entity.getFullyQualifiedName(), fields, ADMIN_AUTH_HEADERS)
|
||||
: getEntity(entity.getId(), null, ADMIN_AUTH_HEADERS);
|
||||
assertListNull(entity.getOwner(), entity.getTests());
|
||||
fields = "owner,tests";
|
||||
fields = "owner,tests,tags";
|
||||
entity =
|
||||
byName
|
||||
? getEntityByName(entity.getFullyQualifiedName(), fields, ADMIN_AUTH_HEADERS)
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>openmetadata-shaded-deps</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<artifactId>elasticsearch-deps</artifactId>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>openmetadata-shaded-deps</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<artifactId>opensearch-deps</artifactId>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<artifactId>openmetadata-shaded-deps</artifactId>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
|
||||
@ -34,6 +34,14 @@
|
||||
"executableEntityReference": {
|
||||
"description": "FQN of the entity the test suite is executed against. Only applicable for executable test suites.",
|
||||
"$ref": "../../type/basic.json#/definitions/fullyQualifiedEntityName"
|
||||
},
|
||||
"tags": {
|
||||
"description": "Tags for this TestSuite",
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "../../type/tagLabel.json"
|
||||
},
|
||||
"default": null
|
||||
}
|
||||
},
|
||||
"required": ["name"],
|
||||
|
||||
@ -0,0 +1,27 @@
|
||||
{
|
||||
"$id": "https://open-metadata.org/schema/entity/configuration/limitsConfiguration.json",
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "FernetConfiguration",
|
||||
"description": "This schema defines the Limits Configuration.",
|
||||
"type": "object",
|
||||
"javaType": "org.openmetadata.schema.configuration.LimitsConfiguration",
|
||||
"properties": {
|
||||
"className": {
|
||||
"description": "Class Name for authorizer.",
|
||||
"type": "string",
|
||||
"default": "org.openmetadata.service.limits.DefaultLimits"
|
||||
},
|
||||
"enable": {
|
||||
"description": "Limits Enabled or Disabled.",
|
||||
"type": "boolean",
|
||||
"default": false
|
||||
},
|
||||
"limitsConfigFile": {
|
||||
"description": "Limits Configuration File.",
|
||||
"type": "string",
|
||||
"default": "limits-config.yaml"
|
||||
}
|
||||
},
|
||||
"required": ["enable"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
@ -44,7 +44,11 @@
|
||||
"EditLifeCycle",
|
||||
"EditKnowledgePanel",
|
||||
"EditPage",
|
||||
"DeleteTestCaseFailedRowsSample"
|
||||
"DeleteTestCaseFailedRowsSample",
|
||||
"Deploy",
|
||||
"Trigger",
|
||||
"Kill",
|
||||
"GenerateToken"
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
@ -0,0 +1,20 @@
|
||||
{
|
||||
"$id": "https://open-metadata.org/schema/system/limitsResponse.json",
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "LimitsResponse",
|
||||
"description": "Limits Config schema",
|
||||
"type": "object",
|
||||
"javaType": "org.openmetadata.schema.system.LimitsConfig",
|
||||
"properties": {
|
||||
"enable": {
|
||||
"description": "Limits Enabled",
|
||||
"type": "boolean",
|
||||
"default": false
|
||||
},
|
||||
"limits": {
|
||||
"description": "Limits",
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
@ -135,6 +135,18 @@
|
||||
"items": {
|
||||
"$ref": "#/definitions/testSuiteConnection/resultSummary"
|
||||
}
|
||||
},
|
||||
"domain" : {
|
||||
"description": "Domain the test Suite belongs to. When not set, the test Suite inherits the domain from the table it belongs to.",
|
||||
"$ref": "../type/entityReference.json"
|
||||
},
|
||||
"tags": {
|
||||
"description": "Tags for this test suite. This is an inherited field from the parent entity if the testSuite is native.",
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "../type/tagLabel.json"
|
||||
},
|
||||
"default": null
|
||||
}
|
||||
},
|
||||
"required": ["name"],
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<parent>
|
||||
<artifactId>platform</artifactId>
|
||||
<groupId>org.open-metadata</groupId>
|
||||
<version>1.5.0-SNAPSHOT</version>
|
||||
<version>1.4.3</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
|
||||
@ -95,7 +95,7 @@ export const removeReviewer = (entity) => {
|
||||
export const deleteGlossary = (glossary) => {
|
||||
cy.get('.ant-menu-item').contains(glossary).click();
|
||||
|
||||
cy.get('[data-testid="manage-button"]').click();
|
||||
cy.get('[data-testid="manage-button"]').scrollIntoView().click();
|
||||
|
||||
cy.get('[data-testid="delete-button"]').scrollIntoView().click();
|
||||
|
||||
|
||||
@ -160,7 +160,10 @@ const removeAssetsFromGlossaryTerm = (glossaryTerm, glossary) => {
|
||||
const deleteGlossaryTerm = ({ name, fullyQualifiedName }) => {
|
||||
visitGlossaryTermPage(name, fullyQualifiedName);
|
||||
|
||||
cy.get('[data-testid="manage-button"]').should('be.visible').click();
|
||||
cy.get('[data-testid="manage-button"]')
|
||||
.scrollIntoView()
|
||||
.should('be.visible')
|
||||
.click();
|
||||
cy.get('[data-testid="delete-button"]')
|
||||
.scrollIntoView()
|
||||
.should('be.visible')
|
||||
@ -198,7 +201,10 @@ const goToAssetsTab = (
|
||||
) => {
|
||||
visitGlossaryTermPage(name, fqn, fetchPermission);
|
||||
|
||||
cy.get('[data-testid="assets"]').should('be.visible').click();
|
||||
cy.get('[data-testid="assets"]')
|
||||
.scrollIntoView()
|
||||
.should('be.visible')
|
||||
.click();
|
||||
cy.get('.ant-tabs-tab-active').contains('Assets').should('be.visible');
|
||||
};
|
||||
|
||||
|
||||
@ -12,7 +12,5 @@
|
||||
*/
|
||||
|
||||
module.exports = {
|
||||
plugins: {
|
||||
autoprefixer: {},
|
||||
},
|
||||
plugins: [require('autoprefixer')],
|
||||
};
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user