Mayur Singal
c758d83276
Fix #9058 : Add time elapsed in seconds instead of miliseconds ( #9310 )
2022-12-16 08:11:30 +00:00
Sriharsha Chintalapani
e42e7e41d8
remove norms for ES index mappings ( #9334 )
2022-12-16 07:43:13 +00:00
Nahuel
819001182f
Fix#9251: DB2 connection config and ingestion update ( #9322 )
...
* DB2 connection config and ingestion update
* Update ingestion/src/metadata/ingestion/source/database/common_db_source.py
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* Update ingestion/src/metadata/ingestion/source/database/common_db_source.py
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* Update bootstrap/sql/com.mysql.cj.jdbc.Driver/v007__create_db_connection_info.sql
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-12-16 07:43:18 +01:00
Ayush Shah
a633befe2a
Fix: Snyk Severity for Delta Spark ( #9323 )
2022-12-16 12:12:44 +05:30
Pere Miquel Brull
d2deb38766
Fix edge nodes ( #9274 )
2022-12-15 14:04:59 +01:00
Onkar Ravgan
b539b299ee
Integrated schema parsers ( #9305 )
...
* Integrated schema parsers
* Addressed review comments
* fixed pytests
2022-12-15 16:54:55 +05:30
NiharDoshi99
c2221fa596
Fix: Datalake s3 get list of files ( #9294 )
...
* Fix: changed func to get list of files
* Fix: minor change
* Fix: minor change
* Fix: adding exception
2022-12-15 14:33:25 +05:30
Mayur Singal
8fe3b09d21
Add Support for Endpoint URL in Datalake ( #9293 )
2022-12-15 07:30:32 +00:00
Mayur Singal
099853ab10
Fix Clickhouse Comments ( #9295 )
2022-12-14 21:42:06 +05:30
Ayush Shah
a6ae9fd11a
Add Test Suite Implementation for Datalake ( #9235 )
2022-12-14 21:14:51 +05:30
Sriharsha Chintalapani
38074f763b
Fix #8509 : Add schema fields ( #9209 )
...
* Fix #8509 : Add schema fields
* Fix #4675 : Ingestion deployment from UI is broken
* Added sample data for topics
* Fixed pytests
* Address comments
* Refactored sampledata according to new schema
* Added return type
* Feat(ui) ✨ : Add support for Avro editor (#9224 )
* Feat(ui) ✨ : Add support for Avro editor
* chore : add mock data for nested fields
* style: add group and opacity css
* add schema fields component to topic details and unit test
* Add locale keys
* Add support for edit description.
* refactor: expandableConfig
* test: add unit test for util method
* chore : make changes according to the schema change
* test : fix unit test and add new mock data
* chore : rename files
* Add row key
* chore : add default value for tags
* chore : update util method
* chore : add support for editing field tags
* chore : rename util files
* test : add unit test
* add comments
* addressing comments
* Address comments
* Added avro requirments
* Added requirement versions
* fixed versions
* protobuf version fix
* chore : rename util test file
* Fixed Dataype
* test: add unit test for schema component
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-12-14 19:56:37 +05:30
NiharDoshi99
b959709275
Fix: added datalake gen2 ( #9245 )
...
* Fix: added datalake gen2
* Fix: pytest
* Fix: pytest
* Fix: made changes to incoporate azure in datalake
* Fix: adlfs version
* Fix: adlfs version
* Fix: add tsv file type
* Fix: refractor for reading tsv file
* Fix: refractor for reading tsv file
2022-12-14 17:05:59 +05:30
Nahuel
0eb49c6100
Use ES call to get table entity ( #9273 )
...
* Use ES call to get table entity
* Address PR comments
2022-12-13 18:32:48 +00:00
Milan Bariya
c6b98751af
Logic improved for BYTES column ( #9261 )
...
* Logic improved for BYTES column
* Try Except block added
* cahange based on comments
2022-12-13 20:10:45 +05:30
Teddy
c7d0fede2f
Fixes #8184 Athena Profiler ( #9270 )
...
* fix(profiler): Athena Median
* fix: linting
2022-12-13 12:03:22 +00:00
Teddy
d1a739ec55
Fixes #9025 -- Added deletion of WebAnalytics events in dataInsight Workflow ( #9208 )
2022-12-13 11:43:29 +01:00
Teddy
387cd570f7
Fixes 9192 - Add entity link in response ( #9239 )
...
* staging commit
* updated terms to french
* fix(dataInsight): updated ES response to get entityType and total non tiered entities
* fix(dataInsight): removed fields=*
* fix(dataInsight): Added entity migration
* cleaned up branch
* feat: Added entity URL for most viewed entity
* feat(dataInsight): Added href to entity
* feat(DataInsight): Added backend logic for entity href
* Fix conflict resolving typos
* Update openmetadata-service/src/main/resources/json/data/dataInsight/mostViewedEntities.json
Co-authored-by: Nahuel <nahuel@getcollate.io>
* fix: addressed comments from PR review
Co-authored-by: Nahuel <nahuel@getcollate.io>
2022-12-13 10:39:09 +00:00
Onkar Ravgan
8ca555d959
Fix #8944 : Added pagination logic for PowerBi ( #9263 )
...
* Added pagination logic
* Added optional keyword
2022-12-13 10:29:09 +01:00
Nahuel
9a4e3a7a46
Fix#8424: Remove brackets from tables and schemas on lineage ( #9257 )
...
* Refactor LineageRunner use
* Address PR comments
* Address pylint errors
* Fix failing test
* Remove brackets from tables and schemas on lineage
2022-12-13 06:40:37 +01:00
Pere Miquel Brull
c75ba751b7
Fix #9116 & #8284 - Clean tableau source, fix ownership, add description and SSL verification ( #9241 )
...
Fix #9116 & #8284 - Clean tableau source, fix ownership, add description and SSL verification (#9241 )
2022-12-13 06:36:55 +01:00
Pere Miquel Brull
9a955036cf
Get sample data for e2e tests ( #9252 )
2022-12-12 15:08:01 +01:00
Sriharsha Chintalapani
145e9d48f6
Fix #9227 : Fix metabase connector to work with non-admin user ( #9228 )
...
* Fix #9227 : Fix metabase connector to work with non-admin user
* Fix #9227 : Fix metabase connector to work with non-admin user
* fix pycheck
* fix pycheck
* fix pycheck
2022-12-12 07:56:58 +01:00
Ayush Shah
2e1fb96751
Add LRU to ES fqn ( #9233 )
2022-12-12 06:46:15 +00:00
Pere Miquel Brull
0963eac48e
Fix #9182 - Airflow Lineage Operator & Airflow lineage state ( #9206 )
2022-12-10 19:54:41 +01:00
Ayush Shah
231b28fc87
Fix 7278: add test connection ( #9196 )
...
* Add Test Connection to Profiler
* remove Code Lint
* Fix AttributeError
* Fix Pytest
* Fix Bigquery Partition tests
* Fix Lint
2022-12-10 17:16:28 +01:00
Teddy
4a5cb870ec
Fixes #9192 - Update ES response for DI ( #9221 )
...
* staging commit
* updated terms to french
* fix(dataInsight): updated ES response to get entityType and total non tiered entities
* fix(dataInsight): removed fields=*
* fix(dataInsight): Added entity migration
* cleaned up branch
2022-12-09 18:53:17 -08:00
Pere Miquel Brull
b1565b8337
Fix #9181 - Handle failed callback status without workflow ( #9193 )
...
* Handle failed status without workflow
* Add log
2022-12-09 10:42:24 +01:00
Milan Bariya
6f12c971e1
Add: function to get different catalogs ( #9179 )
...
* Add: function to get different catalogs
* Change Based on comments
* Change Based on comments
* Change Based on comments
2022-12-08 15:42:58 +05:30
NiharDoshi99
96e77af8a4
Fix : #1308 Docs and Atlas Connector Improvement ( #9072 )
...
* Fix: docs changes
* Fix: changes as per comment
* Fix: atlas connector
* Fix: as per comments
* Fix: as per comments
2022-12-08 12:14:15 +05:30
Teddy
ac77f33b08
Fixes #7447 -- Add freshness metrics to profiler ( #9159 )
...
* refactor(profiler): integrated getter func.
Removed metric getter function from their own file.
Added metric getter to their own interface classs.
created dispatch by value methdo to dispatch metric getter func.
* feature(profiler): added systemProfiler schema
* feat(profiler): workflow fresh. & snflk impl.
* feat(profiler): freshness endpoint for put and get
* feat(profiler): added system met. for redshift
* feat(profiler): freshness met. for bigquery
* fix(profiler): keyword not found in func
* feat(profiler): Added sample data for freshness
* fix(profiler): fetch previous day for BQ
* fix(profiler): sonar + data fetching logic
* fix: typo in SystemMetric Class
* fix: linting
* fix: extracted out EntityList class into models.py
2022-12-07 14:33:30 +01:00
Sriharsha Chintalapani
25449001ca
Fix #9040 : Remove fields such as tableQueries, tableProfile, tests, sample data as part of table fields ( #9041 )
2022-12-06 21:07:04 -08:00
Suresh Srinivas
16d97d6f96
Fix typos in JSON schemas and follow standard naming conventions ( #9183 )
2022-12-06 20:50:34 -08:00
Teddy
f86186d5b0
Fixed time field type for profiler ( #9177 )
2022-12-06 18:30:48 +01:00
07Himank
b5b29c93f5
Track new primary key ( #9145 )
2022-12-06 09:13:07 +05:30
Mayur Singal
81cbf93df5
Fix #9139 : Fix shcema filter being added twice to status ( #9148 )
2022-12-05 15:05:55 +00:00
Ayush Shah
2efa720f59
Fix E2E: new envs ( #9143 )
2022-12-05 19:15:12 +05:30
Pere Miquel Brull
58f3c01470
Fix #7056 - Automatic lineage from ML Model to ML Feature Data Source ( #9113 )
2022-12-05 16:43:03 +05:30
NiharDoshi99
e0a3f3481b
Fix: aggregate type ( #9138 )
2022-12-05 14:50:37 +05:30
Sriharsha Chintalapani
14d82ee7b2
Fix #6463 : Add support for default suggestions for suggestion API ( #9090 )
2022-12-02 10:31:37 -08:00
Pere Miquel Brull
1b3ff505c2
Fix #8858 - Add chart description and add lineage flexibility ( #9124 )
...
Fix #8858 - Add chart description and add lineage flexibility (#9124 )
2022-12-02 16:22:09 +01:00
Onkar Ravgan
23468fb868
Fixed Broken docs links ( #9125 )
2022-12-02 15:42:06 +01:00
NiharDoshi99
ead5ce7227
Fix: dagster keyerror solved ( #9099 )
2022-12-01 18:43:11 +05:30
Pere Miquel Brull
7d2eab332d
Lineage Backend example and fix ( #9095 )
2022-12-01 13:09:36 +01:00
Milan Bariya
2f9f169de9
Fix: Safer default Mark Deleted Tables ( #9065 )
...
* Fix: Safer default Mark Deleted Tables
* Fix: Change Based On Comments
* Fix: Change Based On Comments
* Add: Migration Files
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-12-01 16:26:43 +05:30
Nahuel
00375341f7
Fix#7690: Use ES to build Tables FQN when processing Lineage ( #9079 )
...
* Use ES to build Tables FQN when processing Lineage
* Fix pylent check
* Fix test failing
* Remove warning when calling ES with empty query
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-12-01 09:56:42 +01:00
Nahuel
76773e69de
Fix#6203: Refactor LineageRunner use ( #9073 )
...
* Refactor LineageRunner use
* Address PR comments
* Address pylint errors
* Fix failing test
2022-11-30 16:02:21 +01:00
Ayush Shah
2b2eadf611
Fix Credentials sent via env ( #9071 )
...
* Fix Credentials sent via env
* add new env var
* fix lint
* Add Comment
2022-11-30 15:37:24 +01:00
Abhishek Pandey
e8edf70374
removed-whitespace-from-postgres-and-redshift ( #8927 )
2022-11-30 13:16:24 +05:30
Pere Miquel Brull
6a3ce624b7
Fix #9008 - Add Airflow callback, introduce Workflow Status Mixin and remove retries ( #9049 )
...
Fix #9008 - Add Airflow callback, introduce Workflow Status Mixin and remove retries (#9049 )
2022-11-30 08:30:45 +01:00
Sriharsha Chintalapani
125fefcf6d
Fix logging metadata webhook event ( #9067 )
2022-11-29 20:57:43 -08:00