yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-31 02:29:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,395 Commits 865 Branches 177 Tags
Commit Graph

1489 Commits

Author SHA1 Message Date
Ayush Shah
4a0efe041f
Added connect_args in sql_source (#1791) 2021-12-16 12:33:20 +05:30
Ayush Shah
06712fa425
Metabase and Snowflake Fixed (#1788) 
* Metabase and Snowflake Fixed

* Removed DBT
 2021-12-16 11:57:43 +05:30
Ayush Shah
07924f80c3
Added Metabase Connector (#1726) 
* Added Metabase Connector

* Metabase Added support for lineage

* Modified Metabase as per comments

* Metabase filter patterns added

* Update databaseService.json
 2021-12-15 09:53:55 -08:00
Ayush Shah
90618342cb
Team Description added (#1776) 2021-12-15 09:53:41 -08:00
Sriharsha Chintalapani
92a8ecd0ed
Issue-1762: User profile updates should be only allowed by admin or bots (#1765) 
* Issue-1762: User profile updates should be only allowed by admin or bots
 2021-12-15 01:10:44 -08:00
Alberto Miorin
5c56d479d8
Fix #1749: Location is missing some fields (#1751) 2021-12-14 11:55:13 -08:00
Sriharsha Chintalapani
5a8c70bbe9
Fix #1747: Ingestion: parse connection config method (#1748) 
* Fix #1747:  Ingestion: parse connection config method

* Fix #1747:  Ingestion: parse connection config method

* Fix #1747:  Ingestion: parse connection config method
 2021-12-15 00:58:57 +05:30
Sriharsha Chintalapani
c6ef94cac7
Fix #1733: Ingestion: ElasticSearch index update should use changeLog… (#1736) 
* Fix #1733: Ingestion: ElasticSearch index update should use changeLog timestamp
 2021-12-14 09:49:25 -08:00
Sriharsha Chintalapani
e3d1a95d2c
Fix #1737: Add separate filter patterns for database/schema and tables (#1739) 
* Fix #1737: Add separate filter patterns for database/schema and tables
 2021-12-14 08:50:05 -08:00
Alberto Miorin
93276079ec
Fix #1729: Connector Trino works with username. (#1732) 
Implementation details
I have decided to rename schema_name to database and make it mandatory. Without database there is an error while scanning all available tables. The connector doesn't support multiple databases at the moment. It has to be tested with passwords. Trino requires SSL if you use passwords. It has to be tested with impersonation. I have removed quote_plus because I don't think it's needed.

- [x] Support username
- [ ] There is an integration test
- [ ] Support impersonation
- [ ] Support passwords
- [ ] Support tokens
- [ ] Support multiple databases
 2021-12-13 22:05:28 +01:00
Ayush Shah
14982501c6
Snowflake Version Upgrade and Connection string change (#1707) 2021-12-12 08:26:15 -08:00
Sriharsha Chintalapani
a9198c72f2
Fix #1690: Ingestion: Apply DBT models to views (#1691) 
* Fix #1690: Ingestion: Apply DBT models to views

* Fix #1690: Ingestion: Apply DBT models to views
 2021-12-12 20:53:08 +05:30
Pere Miquel Brull
a7c8056bf5
[issue-1079] - MlFlow Connector (#1677) 
* Prepare infra

* Store experiment information in MySQL & MinIO

* Use CreateMlModelEntityRequest instead of MlModel for PUT operations

* Update MlFlow infra

* Prepare MlFlow source

* Prepare Mlflow workflow

* Simplify test and prepare README

* Revert compose

* Fix compose

* Prepare warnings and fix features

* Use non-default port for integration test

* Use mlflow-skinny for the client application
 2021-12-11 18:40:19 -08:00
Akash Jain
d0d3fce332
fix: add container_name to docker-compose (#1639) 
* adding container_name

* fixed typo --run to --start

* updated container_name prefixing openmetadata_

* updated container_name to prefix openmetadata_
 2021-12-10 08:01:08 -08:00
Matt
3831bb9e4d
Implement s3 bucket policy ingestion (#1570) 
- Fix broken json schema policies (minItems in array instead of minLength)
- Amend s3 ingestion to create policies
- Amend PolicyResource and PolicyRepository to support Policy Rules
 2021-12-10 16:19:43 +01:00
Sriharsha Chintalapani
7b637d4628
Fix #1665: Oracle connector add an option to configure oracle service name (#1666) 
* Fix #1665: Oracle connector add an option to configure oracle service name

* Fixed removal of semi colon

Fixes SQL Command Warning while ingesting

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
 2021-12-10 16:37:19 +05:30
Alberto Miorin
ec7ff6fa13
pip install openmetadata instead of pip install openmetadata-ingestion (#1669) 2021-12-10 10:55:35 +01:00
Sriharsha Chintalapani
6f0ef68701
Fix #1392: parse exclude/include filters from UI generated config (#1634) 2021-12-10 15:17:43 +05:30
Sriharsha Chintalapani
c8c57de37e
Issue 1658: Ingestion changes to add dbtModel as part of Table entity (#1659) 
* Fix #1658: Ingestion changes to add dbtModel as part of Table entity

* Fixes #1652 Remove DBTModel as top level entity and capture information from DBT in existing Table entity

Co-authored-by: sureshms <suresh@getcollate.io>
 2021-12-09 20:32:17 -08:00
Pere Miquel Brull
af134d4afc
Simplify service create or update (#1648) 2021-12-09 17:08:39 +01:00
Ayush Shah
65fb71f819
Update setup.py - removed upper click version (#1638) 
fix #1636 
Removed click upper limit version as upgraded click package doesn't conflict if installed with openmetadata
 2021-12-09 07:58:09 -08:00
Ayush Shah
d5e13a9d22
pymssql and pyodbc support added (#1625) 2021-12-08 08:06:13 -08:00
Ayush Shah
7263510124
Query Log fixed (#1615) 
* Query Log fixed

* Wrong ColumnName fixed

* Added Service Name in usage configs
 2021-12-08 07:36:45 -08:00
parthp2107
64386b035e
added new tests for tags page (#1621) 
* added new tests for tags page

* addressing reviewdog findings

* Added Sample-usage service_name

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2021-12-08 16:54:48 +05:30
Ayush Shah
63ea123b43
Sample User parameters modified (#1622) 2021-12-08 14:53:30 +05:30
Sriharsha Chintalapani
f33a08377c
Fix #1618: Fix usage query parsing (#1619) 2021-12-07 23:53:53 -08:00
Ayush Shah
67433f559c
Trino JDBC updated (#1614) 2021-12-08 13:14:38 +05:30
Pere Miquel Brull
0a716a49a9
[issue-1484] - Deprecate ingestion Common Record (#1608) 
* Refactor Sink to Generic Entity

* Refactor Source to Generic Entity

* Refactor Processor & Stage to Generic Entity

* Deprecate common Record

* Clean unused imports

* Scope import
 2021-12-07 18:00:59 -08:00
Ayush Shah
777afd1d3f
Openmetadata.yaml for docker updated (#1606) 
* Dockerfiles changed - updated elasticsearch, openmetadata.yaml

* Added blue color to docker info
 2021-12-07 11:27:16 -08:00
Ayush Shah
b592f86941
Added Unit tests for datatypes (#1405) 
* Added Unittests for datatypes

* OpenMetadata Datatypes added in unittest

* Added few more datatype support

* SqlTypes updated
 2021-12-08 00:31:51 +05:30
Ayush Shah
1eae6f39ab
fix 1596, fixed indexing of DBT models (#1600) 2021-12-07 16:37:02 +05:30
Alberto Miorin
8621ee3fa0
Fix #1533, #1522: (#1575) 
- Glue works with columns containing dots.
- Glue works with temporary credentials.
- Fix bug in TableRepository, the EntityType was database instead of table.
- only region_name is mandatory in Glue config.
- first integration test for Glue. The CI integration with AWS is still missing.
 2021-12-06 16:03:05 -08:00
Sriharsha Chintalapani
3cc892769a
Fix #1451: Avoid printing password values into log files during ingestion (#1452) 
* Fix #1451: Avoid printing password values into log files during ingestion

* Update cmd.py - removed printing of config

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2021-12-06 14:18:13 -08:00
Sriharsha Chintalapani
ddf4f13c22
Fix #1563: Filter by database name (#1580) 2021-12-06 12:17:24 -08:00
Ayush Shah
3bcbada155
Fix Sample Data ingestion (#1583) 2021-12-07 00:36:50 +05:30
Ayush Shah
85f132738b
Remove faker from Sample-data (#1536) 
* Removed Faker as a dependency

* Added ModuleNotFoundError

* Removed faker from requirements.txt

* Removed SampleData plugin from setup.py
 2021-12-06 10:22:27 -08:00
Pere Miquel Brull
185ef5e98f
Get id as str or uuid (#1562) 2021-12-06 08:40:53 +01:00
Sriharsha Chintalapani
4206f4d119
Fix #1558: Fix metadata ingestion to remove service, database fields from api calls (#1559) 2021-12-05 20:32:17 -08:00
Sriharsha Chintalapani
2d348e9704
Add SSL options to ElasticSearch client - Ingestion & Server (#1548) (#1557) 
* Add SSL options to ElasticSearch client - Ingestion & Server (#1548)

* Add SSL options to ElasticSearch client - Ingestion & Server

* Add SSL options to ElasticSearch client - Ingestion & Server

* Issue-1556: Support for ElasticSearch SSL , Bootstrap command create and delete, ingestion updates for change descriptions

* Issue-1556: Fix checkstyle issues

* Issue-1556: Fix checkstyle issues
 2021-12-04 18:54:36 -08:00
Sriharsha Chintalapani
63a93c8944
Fix #1356: Add new entities to ES index from OpenMetadata server (#1423) 
* Fix #1356: Add new entities to ES index from OpenMetadata server

Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
 2021-12-04 08:45:17 -08:00
Ayush Shah
128a69045d
Fix 1477: Bigquery Usage - queryConfig (#1478) 
* Fix 1477: Bigquery Usage - queryConfig

* Bigquery Usage Modified
 2021-12-03 08:57:44 -08:00
Ayush Shah
341d2d5bf8
Update setup.py (#1542) 2021-12-03 08:56:29 -08:00
Alberto Miorin
f9436c65f0
Fix #1306: Evolve Location entity integration (#1442) 2021-12-03 02:13:13 +01:00
Ayush Shah
0e205d93dd
Added DBT to airflow, modified tables.json - added profile data, fixed user's displayName (#1501) 
* User's Name fixed, added profile data, added dbt pipeline under airflow

* service name changed
 2021-12-01 08:32:29 -08:00
parthp2107
a9e70e6a89
License header update (#1498) 
* updated liscense header

* updated license header

* updated license header

* updated license header

* addressing stylecheck findings
 2021-12-01 12:46:28 +05:30
Sriharsha Chintalapani
4ea45d3be3
Fix #1496: Follow-up: fix for sample-data after user classes refactored (#1497) 
* Fix #1496: Follow-up: fix for sample-data after user classes refactored

* Fix #1496: Follow-up: fix for sample-data after user classes refactored
 2021-12-01 09:38:30 +05:30
Sriharsha Chintalapani
4160f209ef
Fix #1490: Fix broken sample-data ignestion due to User classes refactored (#1491) 2021-11-30 14:41:10 -08:00
Sriharsha Chintalapani
a595332701
Fix #1471: Add Intellij idea settings to repo (#1472) 
* Fix #1471: Add Intellij idea settings to repo
 2021-11-30 12:14:54 -08:00
omriAl
96e473b5cb
#1051: migrated all usages of models.MetadataUser & models.User classes to g… (#1473) 
* migrated all usages of models.MetadataUser & models.User classes to generated.schema.teams.user.User classes instead.

* formatted file with black

Co-authored-by: omri.alon <omri.alon@myheritage.com>
 2021-11-30 07:57:07 -08:00
Sriharsha Chintalapani
7009a0bc12
Fix #1073: Apache Druid Support (#1424) 
* Fix #1073: Apache Druid Support
 2021-11-29 10:59:15 -08:00