Abhishek Pandey
fde53809eb
added-query-log-parser-in-postgres ( #6750 )
...
added-query-log-parser-in-postgres (#6750 )
2022-08-29 06:48:29 +02:00
Teddy
ce578e73d4
Fixes #5831 by implenting testSuite workflow logic ( #6911 )
...
* Added database filter in workflow
* Removed association between profiler and data quality
* fixed tests with removed association
* Fixed sonar code smells and bugs
* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample
* moved status to workflow
* Fixed tests
* removed test logic from profiler sink
* Added logic to return sample from workflow sample value
* Added profiler examples
* Updated documentation for profiler
* Fixed code smells
* commited changed to profiler
* initial commit of the revamp workflow
* Fixed python formating
* cleaned up profiler submodule by removing test related files and functions
* Added airflow DAG logic for testSuite workflow
* Fixed code smells + added airflow ingestion tests + fixed comments
2022-08-25 10:01:28 +02:00
Teddy
78b5f8c8e2
Part 1 of #5831 -- Profiler workflow implementation ( #6809 )
...
* Added database filter in workflow
* Removed association between profiler and data quality
* fixed tests with removed association
* Fixed sonar code smells and bugs
* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample
* moved status to workflow
* Fixed tests
* removed test logic from profiler sink
* Added logic to return sample from workflow sample value
* Added profiler examples
* Updated documentation for profiler
* Fixed code smells
2022-08-19 10:52:08 +02:00
Onkar Ravgan
424239840f
Added result limits to usage queries ( #6748 )
...
* Added result limits to queries
* Fixed formatting
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-17 08:07:39 +02:00
Abhishek Pandey
a6a7662517
dagster-connector-added ( #6513 )
...
* dagster-connector-added
* code-formatted
* dagster-comments-removed
* dagster-comments-removed
* dagster-unittest-added
* code-formatted-pr-comments-resolved
* improved-code-quality
* increased-py-test-time-out
* improved-dagster-tests
* improved-dagster-tests
* reverted-pytests-timeout
* dagster-pytest-improved
* added-schema-in-omdcore
* updated-schema-in-omd-core
* ui-schema-updated-omd-core
* updated-dagster-schema
* dagter-schema-test-updated
2022-08-13 15:18:14 +05:30
Pere Miquel Brull
99f0c96df4
Fix #6278 - Lineage workflow + Usage workflow ( #6568 )
...
* Fixing conflicts
* Fixing Conflicts
* Fix snowflake
* Prepare snowflake and redshift
* snowflake use a single query
* Add bigquery
* Add clickhouse
* format
* Add query log lineage source
* Add filters to core
* Prepare mssql usage
* Update schemas
* Add example workflows
* Prepare lineage deployment
* Remove lineage
* Use filters
* Prep lineage DAG
* Prepare Lineage Ingestion
* Test lineage workflow
* format
* Comments
* fixed tests
* Revert "fixed tests"
This reverts commit 0e846c7c4806098e4adf7c071d133d1df68aedf5.
* fixed get connection method
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-12 13:24:58 +05:30
Onkar Ravgan
af33cc0a31
Fixed Tableau ingestion issue ( #6671 )
...
* Fixed Tableau ingestion issue
* revmoved doc string
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-09 23:55:32 +05:30
Abhishek Pandey
a08201e383
multiple-database-service-names-added-in-dashboard ( #6591 )
...
* multiple-database-service-names-added-in-dashboard
* code-formatted
* code-smell-removed
* code-formatted
* improved-dbserviceName-handling-in-dashboard
* fix checks failing
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2022-08-08 22:21:28 +05:30
Ayush Shah
a6db2e8a84
Fix for profiler: modified filter patterns and added error handling ( #6608 )
2022-08-08 10:43:17 +05:30
Mayur Singal
5e1b124d57
Fix #6530 & Fix #4975 : Oracle Connection Enhancements ( #6579 )
...
* Fix #6530 & Fix #4975 : Oracle Connection Enhancements
* updated migrations & docs
2022-08-05 10:28:04 +05:30
Mayur Singal
c6efe8c142
Fix #6158 : Fivetran Connector Added ( #6329 )
...
* Fivetran Connector Added
* Fivetran unit test added
* Fixed Unit Tests
* Added Fivetran Docs & Added Fivetran in Pipeline Listing
2022-07-27 20:50:56 +05:30
Abhishek Pandey
f125c597a9
Atlas connector fix ( #4112 )
...
* refactored-atlas-schema
* atlas-source-connector-fixed
* code-formatted
* updated-service-creation
* code-formatted
* atlas-json-updated
* atlas-removed-from-database-service
* atlas-fqn-and-execution-flow-updated
* atlas-connector-fixed
* atlas--amundsen-removed-from-db-service
* code-cleanup
* removed-unused-codes
* code-smelled
* code-smell-removed
2022-07-25 11:23:36 +05:30
Milan Bariya
187de08d1f
dbServiceName field move from serviceConnection to sourceConfig ( #6047 )
2022-07-14 11:25:31 +05:30
Mayur Singal
dc497d298b
Fix #5768 : Clickhouse Client Connection Fix ( #5771 )
2022-06-30 12:14:27 +02:00
Mayur Singal
d7cf1289d4
Yaml Fixes ( #5733 )
2022-06-29 15:25:14 +02:00
Milan Bariya
338d1efe40
FIX 5419: Glue pipeline connector not working ( #5467 )
...
FIX 5419: Glue pipeline connector not working (#5467 )
2022-06-27 11:38:41 +02:00
Pere Miquel Brull
a14de6eee1
Fix #5367 - MlFlow connector & MLServices ( #5446 )
...
Fix #5367 - MlFlow connector & MLServices (#5446 )
2022-06-21 14:54:36 +02:00
Onkar Ravgan
21dac15176
Added Mode Connector ( #5475 )
...
* rebase with main
* Added lineage for mode
* added lineagerunner and enum constants
* added host port
* refactored mode according to new dashboard topology
* checkstyle fixes
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-06-19 23:59:38 +05:30
Abhishek Pandey
e8975aac01
datalake-csv-files-ingestion-added ( #5343 )
...
datalake-csv-files-ingestion-added (#5343 )
2022-06-15 08:57:21 +02:00
Milan Bariya
ab7689d7a8
FIX 5131: Fix Get Pipeline Status from Airflow metadata connector ( #5396 )
...
FIX 5131: Fix Get Pipeline Status from Airflow metadata connector (#5396 )
2022-06-13 17:44:21 +02:00
Mayur Singal
51a275c481
Fix #1121 : Added Airbyte Source ( #5372 )
...
Fix #1121 : Added Airbyte Source (#5372 )
2022-06-13 06:45:25 +02:00
Pere Miquel Brull
8e9d0a73f6
Fix #3573 - Sample Data refactor & ORM converter improvements ( #5265 )
...
Fix #3573 - Sample Data refactor & ORM converter improvements (#5265 )
2022-06-08 16:10:40 +02:00
Milan Bariya
5e3f190f5d
Add: Add databaseSchema filed in Yaml file ( #5316 )
2022-06-07 12:12:41 +05:30
Mayur Singal
b3428771a3
Fix #4767 : Refactor Usage & Lineage ( #5190 )
...
Fix #4767 : Refactor Usage & Lineage (#5190 )
2022-06-03 10:12:28 +02:00
Onkar Ravgan
4671ca00b0
Refactored PowerBi Connector ( #5107 )
...
Refactored PowerBi Connector (#5107 )
2022-06-01 10:36:12 +02:00
Pere Miquel Brull
8e121fb306
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector ( #5125 )
...
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector (#5125 )
2022-05-25 08:35:16 +02:00
Mayur Singal
e1a207a37f
Migration Script Updated Yaml ( #5021 )
2022-05-18 20:03:16 +05:30
Ayush Shah
da7b67173c
Modified jsons to yamls ( #4980 )
2022-05-17 18:12:18 +05:30
Milan Bariya
0acf88e1ba
ISSUE-1072: Add Apache Pinot support ( #4649 )
...
* ISSUE-1072: Add Apache Pinot support
* ISSUE-1072: Add Apache Pinot support
* Rename pinotdbConnection.json to pinotDBConnection.json
2022-05-11 11:13:28 +02:00
Mayur Singal
76e36e812d
0.9 to 0.10 Metadata Migration Script ( #4572 )
...
0.9 to 0.10 Metadata Migration Script (#4572 )
2022-05-10 09:33:27 +02:00
Ayush Shah
a4212ec356
Revamp Snowflake ( #4653 )
...
Revamp Snowflake (#4653 )
2022-05-09 15:54:36 +02:00
Sriharsha Chintalapani
34858f3b32
Topics: Add Sample data ( #4787 )
...
* Topics: Add Sample data
* Topics: Add Sample data
* Topics: Add Sample data
* Topics: Add Sample data
* Topics: Add Sample data
2022-05-09 12:50:54 +02:00
Mayur Singal
fd521763d4
Fix #4590 : Usage through query log file ( #4719 )
...
* Usage through query log file
* Pick fileds if available in file
* Created Usage Souce
2022-05-09 11:37:52 +02:00
Ayush Shah
7c8c51e862
Added consumer config in kafka ( #4365 )
2022-04-23 08:24:25 -07:00
Onkar Ravgan
09f928b01b
fixes to trino connector ( #4377 )
...
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-04-22 09:34:07 -07:00
Mayur Singal
bbf79d3222
Refractor test connection for glue & dynamodb ( #4373 )
...
* Refractor test connection for glue & dynamodb
* Renamed test_connection_client
2022-04-22 17:23:42 +02:00
Pere Miquel Brull
ca0b26dded
Fix #4313 - Add title to connection schemas; Fix #4335 - AWS Credentials ( #4338 )
...
Fix #4313 - Add title to connection schemas; Fix #4335 - AWS Credentials (#4338 )
2022-04-22 11:25:05 +02:00
Ayush Shah
f4e6d08812
Refactor kafka connection ( #4326 )
2022-04-21 21:54:30 +05:30
codingwithabhi
c09dc3bff7
athena-source-connection-fixed ( #4278 )
...
* athena-source-connection-fixed
* ran-core_bump_version_dev
* aws-cred-json-schema-support-added-in-athena
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-04-21 16:07:29 +02:00
Mayur Singal
596f3ed9ca
Bigquery Config Fix ( #4275 )
2022-04-20 16:47:00 +05:30
Ayush Shah
09f01fc7e3
Issue 4237 - fixes empty dictionary ( #4239 )
2022-04-19 21:22:46 +05:30
Ayush Shah
61591058fb
Added support openmetadata extraction - Ingestion ( #4195 )
2022-04-19 19:37:41 +05:30
Pere Miquel Brull
256b16d877
Fix #4032 - Bigquery properties & GCS Credentials ( #4202 )
...
Fix #4032 - Bigquery properties & GCS Credentials (#4202 )
2022-04-19 12:31:34 +02:00
Milan Bariya
924e211ba1
ISSUE-4076: Use the central get_engine method for usage ( #4180 )
...
* ISSUE-4076: Use the central get_engine method for usage
* ISSUE-4076: Use the central get_engine method for usage
* ISSUE-4076: Use the central get_engine method for usage
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2022-04-19 12:35:01 +05:30
pmbrull
0c55a7ccc1
Add database required fields
2022-04-18 17:38:34 +02:00
pmbrull
c7fe4f8968
Add dashboard required fields
2022-04-18 16:45:33 +02:00
pmbrull
3a9d8db03d
Update example
2022-04-18 16:41:18 +02:00
pmbrull
422cfb0976
Fix name
2022-04-18 16:41:13 +02:00
pmbrull
e19d0d9d67
dashboard services use URI as hostPort
2022-04-18 16:41:05 +02:00
codingwithabhi
8115c2ece4
issue-4135: updated-mysql-connection-json ( #4136 )
2022-04-14 14:16:09 -07:00
