1831 Commits

Author SHA1 Message Date
Sachin Chaurasiya
6be3ddb7e2
Fix : UI Bug Fixes #2246 #2248 (#2251) 2022-01-17 23:15:19 +05:30
Akash Jain
6350cbb68b
CI for py-ingestion-core comment workflow (#2252)
Part 1 of the py-ingestion-publish CI revamp. This will comment on PR's opened with schema changes and let the Pull requestor notify of version bump to publish dev package in test.pypi.org. See #2239  and #2240 for reference.
2022-01-17 20:35:45 +05:30
Matt
7181141803
Initialize roles and policies with seed data (#2236)
- Set up initial roles and policies seed data
- Extend EntityRepository to support initSeedData()
2022-01-17 00:31:12 -08:00
Sachin Chaurasiya
1f8c28dde1
Generating Typescript interfaces from Json schema. (#2244) 2022-01-17 13:35:56 +05:30
Sachin Chaurasiya
099ddbbce9
Fix: Fixed #2102 #2103 (#2243) 2022-01-17 12:46:06 +05:30
Shailesh Parmar
94cd991462
Fix: issue-2151 Clicking links in column description open the edit panel and then navigates to the link (#2237) 2022-01-17 12:42:36 +05:30
Sriharsha Chintalapani
b4983764e5
Fix 2232: Presto - connection formatting issues when username/password configured (#2233)
* Fix 2232: Presto - connection formatting issues when username/password configured

* Fix 2232: Presto - connection formatting issues when username/password configured
2022-01-17 11:03:17 +05:30
Matt
276128e8b0
Add MetadataOperations RBAC support for PATCH APIs (#2128)
* Add RBAC for PATCH APIs

* Expand scope to all resources except a few (Policy, User, Role, Team, resources that doesn't support PATCH)

* Fix code smells
2022-01-16 09:24:26 -08:00
Pere Miquel Brull
ebda556adc
[issue-2223] - Lineage Callback for any operator (#2225)
* Fix logging

* Reorganise lineage

* Add callback

* Add metadata config

* Fix type smell
2022-01-15 18:57:48 +01:00
Pere Miquel Brull
2728041d0f
[issue-2213] - Airflow lineage refactor (#2218)
* Fix ometa logging

* Refactor airflow lineage

* Refactor airflow lineage

* Add airflow lineage integration tests

* Fix circular dependency
2022-01-14 13:00:33 -08:00
Alberto Miorin
4fedb5fd9b
Fix #2199: Follow-up of PR: Support for getting deleted entities in GET and LIST operations (#2206)
* Improve tests for deleted entities, fix User->Teams->OwnsEntities

* Improve doc for recursive deletion and improve UsageRepository

* Remove listAfter and listBerfore without Include. Add Include.NON_DELETED to the resources without deletion like Bots, Metrics, and Reports. Improve UserRepository.

* Add method to add all relationships for the get deleted entity test. Adapt the test for the entity User.

* Add soft deletion to Webhook. Implement Include for Webhook.

* Improve existing test to include a check for owner.

* Fix populateService of Table

* Fix Team setFields.
2022-01-14 12:19:19 -08:00
Shailesh Parmar
9a55d4efb5
Fix: issue 2098, 2099 & 2192: Resolve text overflow issue (#2216) 2022-01-14 19:46:42 +05:30
Sachin Chaurasiya
60a28ffef6
Fix: Fixing UI Bugs #2203 #2204 #2205 #2191 #2097 (#2214)
* Fix: Fixing UI Bugs  #2203 #2204 #2205 #2191

* Fixing #2097
2022-01-14 19:10:27 +05:30
Shailesh Parmar
9f5a37f51f
Fix: issue-2178 The 'matches' when searched for any keyword doesn't show appropriate count. (#2212) 2022-01-14 17:44:19 +05:30
Sachin Chaurasiya
3babfd9a61
Fixed : Add bots tab to group users with isBot set to true. (#2210) 2022-01-14 16:44:56 +05:30
Sachin Chaurasiya
5ac9acd983
UI: Adding Support for Manual lineage (#1974)
* UI: Adding Support for Manual lineage

* Minor fix

* Adding validation

* Adding editMode handler

* Adding micro iteraction

* Minor Fixes

* Adding support to clear selected entity

* Removed unwanted props

* Removed editable argument from getLineageData Function.

* Removed EntityLineage Drawer

* Removed unwanted codes

* Adding customControls

* Adding toggler for view/edit mode

* minor fix

* Adding sidebar for entities in editmode

* Adding state for reactflow instance

* Adding support for drag and drop

* Minor fixes to avoid flickerness

* adding iseditmode params to getlineage data helper

* fixing icon styling

* Adding state to hold newAdded nodes

* Fixed sidebar styling

* Adding searchbar for custom node

* Adding support for entity suggestions and selected entity state.

* Adding database name in suggestion for table entity.

* Adding arrow head type to new edge

* Adding addlineage api

* Adding support for real time changes

* Style : making some style changes to controls and widgets.

* Adding support for deleting lineage edges.

* Adding background image for graph.

* Adding support for removing new selected entity.

* Adding micro interaction for add new edge operation.

* Adding micro interacton for delete operation.

* Renamed component and added License.

* Disabling cancel button while deleting edge

* Addressing review comments

* Adding checks for already existing entity  node.

* Fixed new node handler position

* Reseting state when entity is alredy present.

* Changed delete icon for nodes.
2022-01-14 16:30:46 +05:30
Shailesh Parmar
9cb6f3c200
Login page revamp (#2207)
* rewamp login page

* added carousal for login

* addressing comments
2022-01-14 15:40:12 +05:30
Sachin Chaurasiya
a6548bf669
Fixed : UI: rename "dataset" in URL to "table" (#2208) 2022-01-14 14:05:36 +05:30
darth-coder00
afcf725491
Fixed Users stat click and owner name inconsistency (#2200) 2022-01-14 11:47:00 +05:30
Akash Jain
105d8c1b67
Feat: docker cmd improvements (#1895)
* revamped docker commands

* update metadata docker commands in comments

* Added Docker Compose Check

* code refactored to docker.py

* fix py-checkstyle lints

* Adding clean functionality; refactor code

* pylint checkstyle fixes

* version string revised
2022-01-13 20:39:46 -08:00
Suresh Srinivas
5a8e2ee31f
Issue2194 - Change dateTime field to unix epoch timestamp milliseconds and add support for webhook secret key (#2195)
* Add support for webhook secret key

* Fixes #2194 - Change dateTime field to unix epoch timestamp milliseconds and add support for webhook secret key

* Fixes #2194 - Added support for upgrading existing metadata to move dateTime attributes to timestamp
2022-01-13 12:08:42 -08:00
Alberto Miorin
e0423ac16a
Fix #2036: Support for getting deleted entities in GET and LIST operations (#2126)
* Create the first failing test.

* Find deleted location.

* Support getService for a deleted location

* Add getByName for Location

* Add test for list

* Add list of Location

* Refactor getService

* Replace the Java Enum with a JSON type

* Fix null pointer exception in getService when the relationship is recovered while the entity is still deleted.

* Simplify the logic using Include.ALL. Implemented Table.

* Add test for Table deleted and Location non-deleted

* Fix small comment

* Implement add Follower delete Entity.

* Add chart and check container

* Add dashboard

* Add database

* Add mlmodel

* Add pipeline

* Add policy

* Add dashboard

* Add databaseService

* Add messaginservice

* Add pipelineService

* Add storageService

* Add role

* Add team

* Add topic

* Add ingestion

* Fix getFollowers NPE

* Disable Webhook tests for Include

* Disable User tests for Include

* Instead of passing Include.ALL, we pass the flag based on the deleted state of entity.

* 1st clean up of 'deleted = false'

* 2nd clean up of 'deleted = false'

* Delete useless return statement.

* Remove duplicated code

* Remove duplicated code in tests

* Remove duplicated code in EntityDAO

* Fix UserDAO and remove duplicated code in EntityRepository and EntityDAO

* Remove Include.NON_DELETED when possible.

* Rebase and fix the formatting
2022-01-13 18:35:31 +01:00
parthp2107
8e19756764
added test for multiple tags (#2196)
* added tests for multiple tags

* minor changes

* addressing style-check
2022-01-13 20:44:34 +05:30
darth-coder00
b29fdd3ccc
Enhanced Tour feature (#2082)
* Enhanced Tour feature

* Fix: #2182 - Tour steps getting skipped

* Fix: #2180 - Stat links deactivated on browser back from Tour route

* Addressing comments

* Addressing comments

* Adding license
2022-01-13 20:43:07 +05:30
darth-coder00
ad362a31fd
Fixed Search filter issues (#2197) 2022-01-13 19:43:32 +05:30
codingwithabhi
5714c4122b
Move sql queries (#2183)
* sql-queries-moved-separate-file

* sql-queries-moved-separate-file

* redshift-and-neo4j-queries-added

* redshift-and-neo4j-queries-added

* neo4j-query-imported

* code-formatted

Co-authored-by: = <=>
2022-01-13 14:16:54 +05:30
Akash Jain
00ccd612ee
fix: Ingestion Core CI to Publish PyPi packages (#2187)
* Remove gitignore on _versions.py

CI commits and pushes the new version in _versions.py and since the folder is marked under gitignore, CI fails to commit and push changes to the branch.

* bump version to 0.8.0.dev0 as already published
2022-01-13 12:40:47 +05:30
Sriharsha Chintalapani
c4d7c76198
Event publishers to ElasticSearch and Slack (#2118)
* ElasticSearch & Slack Event publishers

* Fix #2165: Support for deleted entities in ElasticSearch
2022-01-12 16:35:27 -08:00
Sriharsha Chintalapani
f9a58b418a
DefaultAuthorizer should update the existing users and make them admin/bot based on config (#2173) 2022-01-11 23:37:13 -08:00
Suresh Srinivas
a2b98db7f2
Fixes #2068 Tag duplication (#2170)
* Restore deleted entity on PATCH requests

* Add support for deleting edge in lineage

* Fixes #2068 Tag duplication
2022-01-11 17:59:22 -08:00
Matt
f323afe0cd
Support role ingestion from sample data (#2167) 2022-01-11 16:51:18 -08:00
Sriharsha Chintalapani
80cc907224
Fix #2168: Fix bootstrap_storage to parse env variables in the server config (#2169)
* Fix #2168: Fix bootstrap_storage to parse env variables in the server config
2022-01-11 16:47:11 -08:00
Suresh Srinivas
4fc9db8847
Issue2147 - Add support for deleting edge in lineage (#2164)
* Restore deleted entity on PATCH requests

* Add support for deleting edge in lineage
2022-01-11 13:45:39 -08:00
Sriharsha Chintalapani
7b24e31973
Fix #2143: Allow environment variables to be substituted in OpenMetadata config (#2146)
* Fix #2143: Allow environment variables to be substituted in OpenMetadata config
2022-01-11 10:13:49 -08:00
darth-coder00
5fa0994354
Updated Typescript interfaces from schema (#2157) 2022-01-11 21:21:31 +05:30
darth-coder00
adb63cc982
Fix: Feeds UI adjustments and existing name restriction (#2154) 2022-01-11 18:54:59 +05:30
parthp2107
cf2381482d
added selenium tests for tags with space (#2129)
* added selenium tests for tags with space

* addressing style check

* addressing comments

* using config for url

* minor changes

* minor changes

* changes for new service page

* empty commit, testing workflow

* minor changes

* addressing style-check
2022-01-11 17:45:28 +05:30
parthp2107
2cb490c294
added test for service with space as name (#2150) 2022-01-11 16:43:42 +05:30
darth-coder00
cbb0b837c0
Fix: Misc issues for adding Tags, Teams and Service (#2149)
* Fix: Misc issues for adding Tags, Teams and Service

* Addressing comments
2022-01-11 15:06:46 +05:30
Ayush Shah
f379b35279
Reflection Cache Implementation (#2016)
* Reflection Cache for Bigquery and Redshift

* Overrided few sqlalchemy packages

* Added Geography Support

* Reformatted files

* DBT models error handling implemented

* Geography type added as a custom sqlalchemy datatype

* GEOGRAPHY and VARIANT added as custom sql types

* Implemented file formatting using black

* Implemented file formatting using black
2022-01-11 14:58:03 +05:30
Sriharsha Chintalapani
cf6f438531
Fix #1994: Add support for marking dataset entities as deleted (#2141)
* Fix #1994: Add support for marking dataset entities as deleted

* Fix #1994: Add support for marking dataset entities as deleted

* Fix #1994: Add support for marking dataset entities as deleted
2022-01-11 11:42:53 +05:30
Pere Miquel Brull
bc05e4945d
update name (#2138) 2022-01-10 08:56:12 -08:00
darth-coder00
464181f074
Fix: Flashy Popover, uneven UI for Tags, Teams and Services page (#2132) 2022-01-10 18:09:05 +05:30
Pere Miquel Brull
d3b6c7cf27
[issue-1973] - Python API from Sklearn to MlModel (#2119)
* Move staticmethods to utils

* Use functions from utils

* Convert sklearn to MlModel

* merge main
2022-01-10 09:36:08 +01:00
Pere Miquel Brull
178315d68a
[issue-2112] - Add helper to get EntityReference in OMeta API (#2114)
* Use entity list from mixin

* Add entity reference helper

* Add tests for retrieving the entity reference

* Add missing space

* Fix shadowing

* Use get entity ref

* Use get entity ref
2022-01-10 09:07:39 +01:00
parthp2107
783cbb46de
added test for tag duplication (#2096)
* added test for tag dublication

* minor changes

* java style-check changes

* minor fixing in tests

* test fixes

* minor change
2022-01-10 11:25:33 +05:30
Matt
321a0e811b
Integrate DefaultAuthorizer with PolicyEvaluator (#2017) 2022-01-09 21:04:10 -08:00
Matt
ed797dc335
Remove unused Guice (#2120) 2022-01-09 21:03:50 -08:00
Pere Miquel Brull
9194b8e4ba
[issue-2108] - Fix PR base and add repo slug (#2109)
* Fix PR base and add repo slug

* Add SCM revision
2022-01-08 13:13:24 +01:00
Pere Miquel Brull
74e9038b92
[issue-997] - More pylint fixes (#2107)
* Update naming and disable false positives

* Rename for naming convention

* Fix linting issues

* Moved pylint to pylintrc

* Moved pylint to pylintrc

* Rename CI

* Check sonar properties

* Fix pull_request_target checkout
2022-01-07 19:50:53 +01:00