yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-04 23:54:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
13,006 Commits 654 Branches 142 Tags
Commit Graph

67 Commits

Author SHA1 Message Date
Imri Paran
b2ce491ff1
MINOR: Add failed rows sample to test case (#15682) 
* add failed sample data

* format

* fixed masking pii data in test failed rows sample

* format

* failedRowsSamples -> failedRowsSample

* failedRowsSamples -> failedRowsSample

* fixed tests

* format

* wip

* added computePassedFailedRowCount to python client

* comment for loggerLevel

* format

* fixed tests

* tests for putting / deleting failed samples

* format

* format

* added test case for pii test

* changed method name to deleteTestCaseFailedRowsSample

* added getComputePassedFailedRowCount
 2024-04-10 17:00:00 +02:00
Sriharsha Chintalapani
8af194193a
Add stack trace while throwing an error to debug (#15522) 2024-03-12 10:07:46 -07:00
Ayush Shah
0d66c0e514
Minor: Add extended glossary (#14918) 
* Add Nested glossary to extended sample data

* Fix lint

* Add Stress test config

* Fix Lint
 2024-01-31 07:13:04 -08:00
Ayush Shah
0d0d820305
MINOR: Add Static Sample Data, Modify Lineage (#14752) 2024-01-17 20:07:49 +05:30
Ayush Shah
9c6d202555
Add Sample data, modify regex pattern (#14467) 2024-01-11 14:23:33 +05:30
Pere Miquel Brull
e16ad3c5dc
MINOR - Clean ingestion ES code and auth providers (#14358) 
* MINOR - Clean ingestion ES code and auth providers

* clean parser

* Clean security config for the client

* Clean security config for the client

* Improve class conversion exceptions

* Fix tests

* Clean up java client

* Clean up java client

* clean parser

* Fix test

* Fix test

* fix NO_AUTH error

* Fix test

* Format

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
 2023-12-19 12:08:48 +01:00
Mayur Singal
8f6e5eed31
Automated Lineage by Query for Python SDK & CLI (#12672) 2023-07-31 19:25:41 +05:30
Sriharsha Chintalapani
9dbaabad44
Adding MD5 hash to fullyQualifiedName and names to simplify DB indexes for lookups and increasing the size of FQN (#11960) 
* Fix fqn hash

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* Add name & fqnHash and remove generated columns from top level

* merge commits

* merge commits

* merge commits

* Fix glossary import/export

* Fix BotResource Tests

* Fix BotResource Tests

* Fix Glossary Term tests

* Fix Glossary Term tests

* Fix Glossary Import/Export tests

* All backend tests are fixed.

* merge with main

* Fix tests and revert local changes

* Fix tests and revert local changes

* Fix NullPointerException for Glossary and Query

* Fix Tests

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2023-06-19 15:43:05 +05:30
Mayur Singal
8ecd88fd1b
Fix #11312: Deprecate ES Reindex on Ingestion and use reindex APIs instead (#11988) 2023-06-19 12:45:18 +05:30
Ayush Shah
c08e805090
Revert Change (#9534) 
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2022-12-28 14:05:18 +01:00
Pere Miquel Brull
a4521fd664
Fix #6562 - Sources have their own package (#9521) 
Fix #6562 - Sources have their own package (#9521)
 2022-12-27 15:00:22 +01:00
Mayur Singal
e7133d1e86
Fix Elasticsearch reindex through CLI (#9528) 
* Fix Elasticsearch reindex through CLI

* Fix Elasticsearch reindex through CLI
 2022-12-27 12:06:05 +01:00
Sriharsha Chintalapani
422eccf6e7
Fix #8738: Improve Search Relevancy (#8739) 2022-11-16 15:43:41 -08:00
Pere Miquel Brull
f33003485e
ES reindex Airflow workflow (#8723) 
* Added support for 'Metadata Service' connectors

* Fixed failing unit tests

* Removed delete button for 'Metadata Service' connector of type OpenMetadataServer

* Code optimization for SelectServiceType component

* Fixed errors arrived due to backend changes related to OpenMetadata Services

* init reindex

* Added additional step for adding Metadata to ES configs while creating ingestion pipeline for Metadata Service

* Add ES reindex

* Add ES reindex

* Format

* Format

* Rename service

* Pick service name from IngestionPipeline

* fix ui conflicts

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
 2022-11-14 18:59:56 +01:00
Ayush Shah
8138628d26
Fix WorkflowConfig api (#7754) 2022-09-27 09:23:20 -07:00
mohitdeuex
cceff48f0d
Default token (#7695) 2022-09-23 20:03:54 +05:30
Mayur Singal
91d9e810ea
Revert Rename on MetadataEs (#6537) 2022-08-03 19:36:09 +05:30
Sriharsha Chintalapani
a056ccd9a0
Fix #5303: Add an option to ElasticSearch connector to re-create indexes (#5304) 2022-06-05 13:47:44 -07:00
Mayur Singal
b3428771a3
Fix #4767: Refactor Usage & Lineage (#5190) 
Fix #4767: Refactor Usage & Lineage (#5190)
 2022-06-03 10:12:28 +02:00
Sriharsha Chintalapani
2730d13bcc
Fix sample data config (#4778) 2022-05-06 22:01:17 -07:00
Sriharsha Chintalapani
d2eef5ec0e
Fix #4758: Add support for yaml configuration in Ingestion connectors (#4759) 
* Fix #4758: Add support for yaml configuration in Ingestion connectors

* Fix #4758: Add support for yaml configuration in Ingestion connectors

* Fix #4758: Add support for yaml configuration in Ingestion connectors

* Fix #4758: Add support for yaml configuration in Ingestion connectors
 2022-05-06 08:12:09 +02:00
Sriharsha Chintalapani
4531190af6
Fix #4637: Support JWT Token generation for bot accounts (#4647) 
* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts

* Fix #4637: Support JWT Token generation for bot accounts
 2022-05-05 15:32:33 +05:30
Ayush Shah
61591058fb
Added support openmetadata extraction - Ingestion (#4195) 2022-04-19 19:37:41 +05:30
Mayur Singal
eec9cb05d1
Metadata to ES Pipeline Fix (#4150) 2022-04-14 14:15:29 -07:00
Ayush Shah
15766e85a6
Sample Usage fixed (#4065) 2022-04-12 14:57:18 +05:30
Pere Miquel Brull
6a6507e754
Fix #3962 - Profiler uses DatabaseSchema & Sample Data fix (#4056) 2022-04-12 13:40:59 +05:30
Sriharsha Chintalapani
2e870669e3 Fix #4042: Ingestion: Sample data ingestion is failing 2022-04-11 16:12:01 -07:00
Sriharsha Chintalapani
4a572713a7 Fix #3865: Service Connection & Schema Name - Fix ES indexes 2022-04-10 09:47:36 -07:00
Mayur Singal
b5f1a8edeb
Fixed Sample Usage (#3950) 2022-04-08 19:25:07 +05:30
Mayur Singal
7292695bd3
Sample Data Fix (#3888) 
* Sample Data Fix
 2022-04-06 18:26:54 +05:30
Pere Miquel Brull
63533eb388
Fix for connectors based on refactoring of schemas V2 (#3870) 
Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-04-05 18:33:25 -07:00
Sriharsha Chintalapani
7b3e459eb3
Fix #3659 Refactor Service Connection String to be specific to per service (#3804) 
* Fix #3659 Refactor Service Connection String to be specific to per service

* Simplify and centralize Airflow Pipeline info for REST (#3740)

* Remove code

* Modified Configs based on refactoring schema (#3816)

* Clean WorkflowContext


Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-04-04 12:46:09 -07:00
Sriharsha Chintalapani
50799227e4
Fix #2817: Remove indexing of Glossary as another Elasticsearch index (#2818) 2022-02-16 20:10:26 -08:00
Suresh Srinivas
28ba1a3c04
Fixes #2760 - Add entities for Glossary and initial API (#2761) 
* Fixes #2760 - Add entities for Glossary and initial API

* Fixing merge issues
 2022-02-15 20:54:46 -08:00
Ayush Shah
19e5f737ff
Usage fix - removed service_name from bulk_sink (#1956) 
* Query Usage fixed

* Usage updated

* Reformatted change

* Usage files reformatted
 2022-01-04 17:12:10 +05:30
Sriharsha Chintalapani
f33a08377c
Fix #1618: Fix usage query parsing (#1619) 2021-12-07 23:53:53 -08:00
Ayush Shah
47a97e7be0 FQN changed from bigquery to bigquery_gcp 2021-11-29 16:57:39 +05:30
Ayush Shah
759574a8de
Ingestion Optimization - Sample Users, Dockerfiles, Removal of Pandas (#935) 
* Sample Profile Data for Sample Tables (#815)

* Sample Profile Data for Sample Tables

* Disabling Profile as Default

* Added Sample Profile Data to 3 additional sample tables

* Sample Tables fixed (#850)

* Pydantic fix, Docker update (#860)

* Setup.py Modified with openmetadata-airflow package, docker update

* Setup.py Modified

* Update setup.py

* Removed Pandas from Sample Data

* Sample Users added under sample data

* Sample User Standalone pipelines and modules removed

* Docker release package updated

* Dockerfile updated, removed redundant files

* Setup.py removed from ingestion src directory

* User Resource failing check resolved

* Modifying Usage Columns Datatype
 2021-10-26 09:14:24 -07:00
Ayush Shah
72e4b8831f
Removing Cron from all configs (#773) 
* Removing Cron from all configs

* Removing Cron from Documentation
 2021-10-14 15:21:47 -07:00
Sriharsha Chintalapani
f7da8045b4
Fix #605: Ingestion: metadata list apis should paginate (#606) 
* Fix #605: Ingestion: metadata list apis should paginate

* Fix #605: Ingestion: metadata list apis should paginate, Addressing review comments
 2021-09-29 00:06:08 +05:30
Ayush Shah
627481f181
Status record Json encoding bug fixed and pandas not found fixed (#584) 2021-09-25 13:54:04 -07:00
Ayush Shah
7652baa00d
Setup.py Refactored, ES port fix (#521) 
* Pylint build failure fixed

* Setup & dependency modified, Data profiler default to False, ES port fix

* Profiler requirements refactored

* Setup.py requirement fix

* openmetadata-ingestion version upgrade
 2021-09-19 13:59:14 +05:30
Sriharsha Chintalapani
4c6c8fd446
Fix #515: Ingestion: Add ES configuration to allow port (#516) 2021-09-17 08:57:41 -07:00
Sriharsha Chintalapani
20a98aca81
Fix #396: Elastic search is not indexing usage for dashboard (#492) 2021-09-15 09:47:14 +05:30
Sriharsha Chintalapani
1c80dc246e
Fix #456: Make PII-Processor optional and independent install (#457) 2021-09-10 10:41:19 +05:30
Suresh Srinivas
328658ebea [WIP] profiler 2021-09-07 22:03:57 -07:00
Ayush Shah
657962bc4f
MSSQL sample-data query fix (#375) 
* MSSQL sample-data query fix

* Query Format as per Database implemented
 2021-09-06 21:03:04 -07:00
Sriharsha Chintalapani
d0dbcc19b7
Fix #401: Merge sample data generation into single connector (#402) 
* Fix #401: Merge sample data generation into single connector

* Path for datasets modified

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2021-09-05 22:35:02 +05:30
Sriharsha Chintalapani
59c74fffcd
Fix #393: ingestion support for indexing dashboard/charts entities (#394) 2021-09-03 14:54:05 +05:30
parthp2107
d3d0ac19d0
Removed serviceType as a configuration option from workflow configs (#371) 
* Removed serviceType as a configuration option from workflow configs

* Removed serviceType as a configuration option from workflow configs

Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
 2021-09-02 19:45:41 +05:30