Vj-L
f3855a1909
Added the default volumes for ElasticSearch ( #9452 )
...
* Added the default volumes for ElasticSearch
* Added docs for ES Volumes
* Updated as per the comments
* Updated as per the comments
* Added the docs for bare-metal upgrade
* Apply suggestions from code review
* Update docker/local-metadata/docker-compose.yml
* Pylint check addressed
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2022-12-21 18:29:49 +00:00
Mayur Singal
e10fff89b4
Messaging Broker Minor Fix ( #9468 )
2022-12-21 17:41:26 +01:00
Sriharsha Chintalapani
b9242c1390
Fix topic indexes and migrations ( #9439 )
...
* Fix topic indexing
* Fix topic db migrations
* Fix topic db migrations
* Fix topic db migrations
* Fix test; add missing topic indexes on python side; fix ranking for tables using usage count
2022-12-21 10:33:12 +01:00
Teddy
9e01fe0636
feat(CLI): added migration logic cmd ( #9437 )
2022-12-21 07:55:18 +00:00
NiharDoshi99
9a3d599f30
Azure datalake minor changes ( #9407 )
2022-12-21 10:28:41 +05:30
NiharDoshi99
c72938085d
Fix:minor changes in atlas ( #9418 )
2022-12-20 20:44:57 +05:30
Teddy
cadfb68826
fix(profiler): bugs in logic to fetch metrics ( #9423 )
2022-12-20 14:23:44 +00:00
Onkar Ravgan
c1fdc59e84
Added DBT Workflow fixes ( #9419 )
...
* Added DBT fixes
* Addressed review comments
2022-12-20 19:06:01 +05:30
Mayur Singal
07aed10a7e
Snowflake E2E Tests ( #8781 )
2022-12-20 18:00:26 +05:30
Ayush Shah
2bf5eb9051
fix 7995: profileSample % and row number ( #9104 )
2022-12-20 14:55:11 +05:30
Pere Miquel Brull
6f66431587
Fix OM backup ( #9395 )
...
Fix OM backup (#9395 )
2022-12-20 08:23:31 +01:00
Mayur Singal
497eca70ab
Fix DBT HTTP source error ( #9397 )
2022-12-19 14:55:31 +01:00
Milan Bariya
a3495c67f0
Databricks improve code and doc update ( #9394 )
...
* Databricks improve code and doc update
* Change base on comments
* Fix pytest fail
2022-12-19 18:53:10 +05:30
Mayur Singal
425a50bc3a
DBT ingestion improvements ( #9383 )
...
* DBT ingestion improvements
* use fqn build instead of join
* fix linting
* fix linting
2022-12-19 06:37:54 +01:00
Milan Bariya
47a1c8015a
Fix: Databricks table type issue ( #9385 )
...
* Fix: Databricks table type issue
* Fix: Databricks table type issue
2022-12-18 15:31:13 +01:00
Mayur Singal
c06dbfd22e
Fix caching issue for BQ ( #9352 )
2022-12-17 09:16:16 -08:00
Milan Bariya
e1d347c714
Minor change in backup function ( #9370 )
...
* Minor change in backup function
* Minor change in backup function
2022-12-17 17:43:28 +05:30
Pere Miquel Brull
3b7ae73473
Airflow e2e integration test ( #9363 )
...
* Prep airflow operator integration tests
* Add integration test to Makefile
2022-12-16 19:52:12 -08:00
Teddy
b3160130f2
Fixe Ownership and Tier Processing ( #9356 )
...
* fix(DI): Owner perc. and tiers
* fix(DI): pylint
2022-12-16 19:51:54 -08:00
Mayur Singal
5164a55341
Fix Mysql e2e test ( #9364 )
2022-12-16 19:39:02 -08:00
Teddy
3856d63f3a
Fixes #5522 by implementing timeout for the profiler ( #9340 )
...
* feat(Profiler): Added timeout logic in the profiler workflow
* feat(Profiler): Added additional input in workflow UI
* fix(Profiler): timeout text blob in UI
* fix(Profiler): Return profile_results outside the context manager
* Fix(Profiler): Added localizer for timeout label and message
* Update ingestion/src/metadata/interfaces/sqalchemy/sqa_profiler_interface.py
Co-authored-by: Nahuel <nahuel@getcollate.io>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2022-12-16 17:01:12 +01:00
Pere Miquel Brull
7aed6d340b
Fix #9276 - Fix dump syntax for backup ( #9341 )
...
* Update docs
* Clean dump
* Clean dump
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2022-12-16 16:59:13 +01:00
Nahuel
a2b34dd0f4
Fix: Update Ingestion docker images and fix python libraries dependencies ( #9342 )
...
* Update Ingestion docker images and fix python libraries dependencies
* Install also apache-airlfow-providers-http
2022-12-16 14:46:25 +00:00
NiharDoshi99
3190ac2516
Fix: making token as optional field ( #9314 )
2022-12-16 14:17:47 +00:00
Milan Bariya
a15600e4e6
Fix: Metadata backup Azure blob issue ( #9328 )
...
* Fix: Metadata backup Azure blob issue
* Fix: Metadata backup Azure blob issue
* Fix: Metadata backup Azure blob issue
* Doc update for Backup
* Doc update for Backup
2022-12-16 18:59:00 +05:30
Ashish Gupta
7c181a5a89
Dbt worflow ( #9302 )
2022-12-16 17:00:28 +05:30
Ayush Shah
50d1538374
Fix failure in GCS datalake ingestions ( #9336 )
2022-12-16 08:37:39 +00:00
Mayur Singal
c758d83276
Fix #9058 : Add time elapsed in seconds instead of miliseconds ( #9310 )
2022-12-16 08:11:30 +00:00
Sriharsha Chintalapani
e42e7e41d8
remove norms for ES index mappings ( #9334 )
2022-12-16 07:43:13 +00:00
Nahuel
819001182f
Fix#9251: DB2 connection config and ingestion update ( #9322 )
...
* DB2 connection config and ingestion update
* Update ingestion/src/metadata/ingestion/source/database/common_db_source.py
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* Update ingestion/src/metadata/ingestion/source/database/common_db_source.py
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* Update bootstrap/sql/com.mysql.cj.jdbc.Driver/v007__create_db_connection_info.sql
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-12-16 07:43:18 +01:00
Ayush Shah
a633befe2a
Fix: Snyk Severity for Delta Spark ( #9323 )
2022-12-16 12:12:44 +05:30
Pere Miquel Brull
d2deb38766
Fix edge nodes ( #9274 )
2022-12-15 14:04:59 +01:00
Onkar Ravgan
b539b299ee
Integrated schema parsers ( #9305 )
...
* Integrated schema parsers
* Addressed review comments
* fixed pytests
2022-12-15 16:54:55 +05:30
NiharDoshi99
c2221fa596
Fix: Datalake s3 get list of files ( #9294 )
...
* Fix: changed func to get list of files
* Fix: minor change
* Fix: minor change
* Fix: adding exception
2022-12-15 14:33:25 +05:30
Mayur Singal
8fe3b09d21
Add Support for Endpoint URL in Datalake ( #9293 )
2022-12-15 07:30:32 +00:00
Mayur Singal
099853ab10
Fix Clickhouse Comments ( #9295 )
2022-12-14 21:42:06 +05:30
Ayush Shah
a6ae9fd11a
Add Test Suite Implementation for Datalake ( #9235 )
2022-12-14 21:14:51 +05:30
Sriharsha Chintalapani
38074f763b
Fix #8509 : Add schema fields ( #9209 )
...
* Fix #8509 : Add schema fields
* Fix #4675 : Ingestion deployment from UI is broken
* Added sample data for topics
* Fixed pytests
* Address comments
* Refactored sampledata according to new schema
* Added return type
* Feat(ui) ✨ : Add support for Avro editor (#9224 )
* Feat(ui) ✨ : Add support for Avro editor
* chore : add mock data for nested fields
* style: add group and opacity css
* add schema fields component to topic details and unit test
* Add locale keys
* Add support for edit description.
* refactor: expandableConfig
* test: add unit test for util method
* chore : make changes according to the schema change
* test : fix unit test and add new mock data
* chore : rename files
* Add row key
* chore : add default value for tags
* chore : update util method
* chore : add support for editing field tags
* chore : rename util files
* test : add unit test
* add comments
* addressing comments
* Address comments
* Added avro requirments
* Added requirement versions
* fixed versions
* protobuf version fix
* chore : rename util test file
* Fixed Dataype
* test: add unit test for schema component
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-12-14 19:56:37 +05:30
NiharDoshi99
b959709275
Fix: added datalake gen2 ( #9245 )
...
* Fix: added datalake gen2
* Fix: pytest
* Fix: pytest
* Fix: made changes to incoporate azure in datalake
* Fix: adlfs version
* Fix: adlfs version
* Fix: add tsv file type
* Fix: refractor for reading tsv file
* Fix: refractor for reading tsv file
2022-12-14 17:05:59 +05:30
Nahuel
0eb49c6100
Use ES call to get table entity ( #9273 )
...
* Use ES call to get table entity
* Address PR comments
2022-12-13 18:32:48 +00:00
Milan Bariya
c6b98751af
Logic improved for BYTES column ( #9261 )
...
* Logic improved for BYTES column
* Try Except block added
* cahange based on comments
2022-12-13 20:10:45 +05:30
Teddy
c7d0fede2f
Fixes #8184 Athena Profiler ( #9270 )
...
* fix(profiler): Athena Median
* fix: linting
2022-12-13 12:03:22 +00:00
Teddy
d1a739ec55
Fixes #9025 -- Added deletion of WebAnalytics events in dataInsight Workflow ( #9208 )
2022-12-13 11:43:29 +01:00
Teddy
387cd570f7
Fixes 9192 - Add entity link in response ( #9239 )
...
* staging commit
* updated terms to french
* fix(dataInsight): updated ES response to get entityType and total non tiered entities
* fix(dataInsight): removed fields=*
* fix(dataInsight): Added entity migration
* cleaned up branch
* feat: Added entity URL for most viewed entity
* feat(dataInsight): Added href to entity
* feat(DataInsight): Added backend logic for entity href
* Fix conflict resolving typos
* Update openmetadata-service/src/main/resources/json/data/dataInsight/mostViewedEntities.json
Co-authored-by: Nahuel <nahuel@getcollate.io>
* fix: addressed comments from PR review
Co-authored-by: Nahuel <nahuel@getcollate.io>
2022-12-13 10:39:09 +00:00
Onkar Ravgan
8ca555d959
Fix #8944 : Added pagination logic for PowerBi ( #9263 )
...
* Added pagination logic
* Added optional keyword
2022-12-13 10:29:09 +01:00
Nahuel
9a4e3a7a46
Fix#8424: Remove brackets from tables and schemas on lineage ( #9257 )
...
* Refactor LineageRunner use
* Address PR comments
* Address pylint errors
* Fix failing test
* Remove brackets from tables and schemas on lineage
2022-12-13 06:40:37 +01:00
Pere Miquel Brull
c75ba751b7
Fix #9116 & #8284 - Clean tableau source, fix ownership, add description and SSL verification ( #9241 )
...
Fix #9116 & #8284 - Clean tableau source, fix ownership, add description and SSL verification (#9241 )
2022-12-13 06:36:55 +01:00
Pere Miquel Brull
9a955036cf
Get sample data for e2e tests ( #9252 )
2022-12-12 15:08:01 +01:00
Sriharsha Chintalapani
145e9d48f6
Fix #9227 : Fix metabase connector to work with non-admin user ( #9228 )
...
* Fix #9227 : Fix metabase connector to work with non-admin user
* Fix #9227 : Fix metabase connector to work with non-admin user
* fix pycheck
* fix pycheck
* fix pycheck
2022-12-12 07:56:58 +01:00
Ayush Shah
2e1fb96751
Add LRU to ES fqn ( #9233 )
2022-12-12 06:46:15 +00:00
