Suresh Srinivas
5f34bd02d3
Fixes #13595 Consolidate changes by a user in a single session to a single change ( #13617 )
...
* getChangeDescription to use entity and update type to decide previous version
* Fixes #13595 - Consolidate changes by a user in a single session to a single change
2023-11-16 18:14:27 -08:00
Teddy
a1a1d9f4d1
fix: GX integration ts ms ( #13984 )
2023-11-15 16:11:52 +01:00
Mohit Yadav
10d8ec84fe
Logs added for Search Indexing and Stats issue fixed ( #13956 )
...
* Logs added for Search Indexing and Stats issue fixed
* Fix uninstall error
* Add error handling
* fix lint
* Push Job Level Exception on top
* disable flaky tests
* Fix Logs not visible in Search
---------
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-11-13 23:39:56 +05:30
Pere Miquel Brull
8c6777f820
Fix #12406 - Ingestion image with python 3.10 ( #13949 )
2023-11-13 15:08:06 +05:30
Onkar Ravgan
a1b967b857
Nifi wrong password fix ( #13950 )
2023-11-13 14:48:07 +05:30
Mayur Singal
b34111ea00
ElasticSearch: Fix ES connection ( #13919 )
2023-11-13 14:10:11 +05:30
Pere Miquel Brull
c742835766
Auto Tagger Application - Preparing the Ingestion Framework ( #13862 )
...
* Prepare the skeleton for generic app registration
* Prepare the skeleton for generic app registration
* Handle app runner
* Prepare the skeleton for generic app registration
* Prepare the skeleton for generic app registration
* Allow deployment
* Fix PII APP
* Fix lint
* Fix PII APP
* Fix PII APP
* Prepare config-based external apps
* Prepare config-based external apps
* Fix lint
* Prepare config-based external apps
* Fix DI errors
* Amend comments
2023-11-13 08:58:38 +01:00
Mayur Singal
367bac9064
Fix #13787 : Add support for ES data types ( #13916 )
...
* Fix #13787 : Add support for ES data types
* fixed tests
---------
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-11-10 20:14:42 +05:30
Pere Miquel Brull
ae969aa4f3
Fix #13660 - Metadata Backup Memory Exhaustion ( #13935 )
2023-11-10 12:21:48 +01:00
Onkar Ravgan
b7e7625071
Convert dbt test result timestamps to millis ( #13903 )
...
* Fixed dbt test res time to millis
* added postgres migration
2023-11-10 16:15:16 +05:30
Mayur Singal
71a67d6bbe
Fix ES search during lineage ingestion ( #13932 )
2023-11-10 15:50:37 +05:30
Pere Miquel Brull
0eacc829a4
Fix #13794 - Add domain support to the Python SDK ( #13931 )
2023-11-10 11:00:06 +01:00
Pere Miquel Brull
7c06116b53
Add deprecation warnings ( #13927 )
2023-11-10 15:17:07 +05:30
Pere Miquel Brull
8891a9a410
Fix #13906 - Fix add_mlmodel_lineage description field ( #13920 )
2023-11-10 15:16:09 +05:30
Pere Miquel Brull
b250cd8808
Fix #13699 - Add separator for Storage Container manifest ( #13924 )
...
* Fix #13699 - Add separator for Storage Container manifest
* Fix #13906 - Fix add_mlmodel_lineage description field
* Add separator
* Add separator
2023-11-10 10:44:47 +01:00
Mayur Singal
4b625f7ba5
Move pandas top level import ( #13926 )
2023-11-10 14:15:14 +05:30
Hung Duong
575a4a0ebc
put correct dataset only ( #13369 )
...
* put correct dataset only
* update code
---------
Co-authored-by: hung.duong <hung.duong@be.com.vn>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-11-09 10:30:43 -08:00
Mayur Singal
4c19bb5a1d
Add support for external profiler workflow ( #13887 )
...
* Add support for external profiler workflow
* pylint
* resolve comments
* resolve comments
* pyformat
* fix code smell
2023-11-09 18:50:47 +05:30
Mayur Singal
a8145a82fa
Fix #13603 : Configurable Sample Data Rows for Profiler ( #13807 )
...
* Fix #13603 : Configurable Sample Data Rows
* Fix #13603 : Configurable Sample Data Rows for Profiler
* fix table config
* support configurable overwriting of sample data
* add support for schema and database profiler configuration
* chore(ui): put sampleDataStorageConfig under advanced config
* fix tests
* py format
* chore(ui): add sampleDataCount in table profiler config
* fix tests
* pylint & tests
* feat(ui): add profiler settings tab in database and database schema page
* chore(ui): show different inputs for profile sample type
* schema changes to make default storange config null
* add unit test
* schema changes to fix api
* update profiler setting schema
* move profiler settings to manage button
* sync locals
* fix(ui): unit tests
* fix tests
* py format
* fix lint
* minor improvements
* chore(ui): update profiler settings schema
* resolve review comments
* pytest
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-11-09 18:49:42 +05:30
Teddy
7f151ca5f7
fix: remove RESULT_SCAN dep. ( #13904 )
2023-11-09 08:13:59 +01:00
Onkar Ravgan
bdfc9226d5
Added cost analysis producer in DI ( #13872 )
2023-11-08 12:04:48 +05:30
Suresh Srinivas
a89e317a2b
Fixes #13863 - Show inherited relationships of an entity ( #13864 )
...
* Fixes #13863 - Show inherited relationships of an entity
* Test failure fixes
* Commenting out invalid python test
2023-11-07 09:11:06 -08:00
Ayush Shah
ec6184d2da
Fix Trino Dialect Import issue ( #13869 )
2023-11-07 12:10:59 +05:30
Ayush Shah
29b7f3aa33
Fixes 13688: Trino Profiler Bugs ( #13853 )
2023-11-06 17:14:04 +05:30
Onkar Ravgan
c7834e74cc
fixed avro recursive record ( #13856 )
2023-11-06 16:27:06 +05:30
Onkar Ravgan
9d58b56a1c
Added stored procedures sample data ( #13838 )
...
* Added stored proc sample data
* Added sp lineage
2023-11-03 14:05:02 +01:00
Teddy
d025e217d6
fix: catch not Either type in workflow and return explicit error message ( #13796 )
2023-11-02 13:02:26 +01:00
Ayush Shah
0a04ce85bb
Add Multilingual Support in EntityLink ( #13826 )
2023-11-02 14:35:22 +05:30
Mayur Singal
feed873782
Add support for partitioned table support for Athena ( #13812 )
2023-11-01 17:13:28 +05:30
Pere Miquel Brull
7dd3874224
MSTR fix either ( #13788 )
2023-10-31 15:56:04 +05:30
Teddy
ba0d1153cd
Fix databricks lineage with unity catalog ( #13786 )
2023-10-31 15:55:04 +05:30
Teddy
10904049e4
fix: handle lower and upper case name ( #13778 )
2023-10-31 09:51:13 +01:00
Andrey Skibunov
3c8ec74d37
Feature: Dashboard MSTR Integration ( #13604 )
...
* Dashboard MSTR Integration
* Format
* Format and linting
---------
Co-authored-by: Andrey.Skibunov <Andrey.Skibunov@life.com.by>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-10-30 12:10:12 +01:00
Tom Bushell
33fe3b5ee5
Fixes #7858 : Moved ingestion related make commands into Makefile in ingestion directory ( #13677 )
...
* Moved more recipes into ingestion/Makefile
* Removed some recipes into ingestion/Makefile and added import statement
* Modified file paths so that 'make generate' works from the ingestion directory
* Modified checks for current directory
* Fixed function names to be in snake case
* Reverted function names back to camel case
* Reverted changes to js_antlr and py_antlr and moved generate command back into root directory Makefile
* Updated run_ometa_integration_testsrecipe in ingestion/Makefile
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-10-30 08:23:47 +01:00
Mayur Singal
2890f66ac1
Fix for docker start command ( #13728 )
2023-10-26 15:04:06 +02:00
Pere Miquel Brull
c6297b9cdf
Fix status handling for new services ( #13707 )
2023-10-25 20:48:14 +02:00
Teddy
452a33b1a0
Fixes Druid Profiler failures ( #13700 )
...
* fix: updated playwrigth test structure
* fix: druid profiler queries
* fix: python linting
* fix: python linting
* fix: do not compute random sample if profile sample is 100
* fix: updated workflow to test on push
* fix: move connector config to category folder
* fix: updated imports
* fix: added pytest-dependency package
* fix: updated readme.md
* fix: python linting
* fix: updated profile doc for Druid sampling
* fix: empty commit for CI
* fix: added workflow constrain back
* fix: sonar code smell
* fix: added secrets to container
* Update openmetadata-docs/content/v1.2.x-SNAPSHOT/connectors/ingestion/workflows/profiler/index.md
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Update openmetadata-docs/content/v1.2.x-SNAPSHOT/connectors/ingestion/workflows/profiler/index.md
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Update ingestion/tests/e2e/entity/database/test_redshift.py
* fix: ran pylint
* fix: updated redshift env var.
* fix: import linting
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-10-25 20:47:51 +02:00
Onkar Ravgan
de544d2dca
added reg record ( #13705 )
2023-10-25 08:34:07 -07:00
Ayush Shah
bfb361dc85
Fix Bigquery lineage Pytests ( #13695 )
2023-10-25 11:15:41 +05:30
Pere Miquel Brull
0b2d7aecfd
Fix table_usage pydantic validation & unbound var ( #13691 )
2023-10-24 11:38:58 +02:00
Ayush Shah
57cb72c26f
Fix Checkstyle ( #13683 )
2023-10-23 15:51:40 +05:30
Iaroslav Frolikov
420da29841
Fixes #13607 : BigQuery lineage ingestion fails when using GcpCredentialsPath authentication config ( #13608 )
2023-10-23 15:42:06 +05:30
Keagan O'Donoghue
74aef36b1e
ISSUE-13517: Added option to explicitly specify backup filename ( #13661 )
...
* ISSUE-13517: Added option to explicitly specify output filename for metadata backup
* format
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-10-22 13:26:12 +02:00
Onkar Ravgan
0f0bccdd45
Converted and fixed pipelinestatus timestamps to milliseconds ( #13670 )
...
* fixed pipelinestatus timestamps in mills
* Added migrations
2023-10-20 09:39:24 -07:00
Teddy
feb52647d2
fix: conditioned call to getColumnTags to columns fields ( #13652 )
2023-10-20 17:50:52 +02:00
Pere Miquel Brull
8cf8720a9d
Clean Airflow Lineage Backend and migrate status to millis ( #13666 )
...
* Clean Airflow Lineage Backend and migrate status to millis
* Format
* chore(ui): update executions startTs and endTs to millis
* Remove lineage providers
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-10-20 15:42:38 +02:00
Pere Miquel Brull
660bf01a5b
Fix Stored Procedures Lineage for multi-db processes ( #13655 )
2023-10-20 09:14:08 +02:00
Teddy
fc335f2aff
fix: limit fields to only required ones ( #13647 )
2023-10-19 16:16:20 +02:00
Ayush Shah
ad86d8969f
Fix E2E failures ( #13648 )
2023-10-19 17:49:02 +05:30
Pere Miquel Brull
255bfb95b1
Remove duplicates from entity_extension_time_series and add the const… ( #13626 )
...
* Remove duplicates from entity_extension_time_series and add the constraing if missing
* Add sort buffer and work mem
* Revert "Add sort buffer and work mem"
This reverts commit fcfff5feb60c9212bb7c1cad34b524dc8c03bfc5.
2023-10-19 12:15:02 +02:00
