* fix(data-quality): table diff
- added handling for case-insensitive columns
- added handling for different numeric types (int/float/Decimal)
- added handling of boolean test case parameters
* add migrations for table diff
* add migrations for table diff
* removed cross type diff for now. it appears to be flaky
* fixed migrations
* use casefold() instead of lower()
* - implemented utils.get_test_case_param_value
- fixed params for case sensitive column
* handle bool test case parameters
* format
* testing
* format
* list -> List
* list -> List
* - change caseSensitiveColumns default to fase
- added migration to stay backward compatible
* - removed migration files
- updated logging message for table diff migration
* changed bool test case parameters default to always be false
* format
* docs: data diff
- added the caseSensitiveColumns parameter
requires: https://github.com/open-metadata/OpenMetadata/pull/18115
* fixed test_get_bool_test_case_param
* fix import issue
* chore: add raise from status on ingestion step in tests
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* feat: indexed test case results
* feat: added indexation logic for test case results
* style: ran java linting
* fix: IDE warnigns
* chore: added test case results migration
* style: ran java linting
* fix: postgres migration column json ref
* empty commit to trigger queued
* chore: extracted test case results to its own resource
* chore: fix failing tests
* chore: move testCaseResult state from testSuite and testCase to dynamic field fetched from test case results search index
* chore: clean up test case repository
* style: ran java linting
* chore: removed testCaseResultSummary and testCaseResult state from db
* fix: test failures
* chore: fix index mapping type for result value
* chore: fix test failure
* GEN-1493 - Fix opensearch pagination
* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
* GEN-1410 - Improve PII Logging information
* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
* allow_module_level=True
* tests(datalake): use minio
1. use minio instead of moto for mimicking s3 behavior.
2. removed moto dependency as it is not compatible with aiobotocore (https://github.com/getmoto/moto/issues/7070#issuecomment-1828484982)
* - moved test_datalake_profiler_e2e.py to datalake/test_profiler
- use minio instead of moto
* fixed tests
* fixed tests
* removed default name for minio container
* fix: Allow non numeric numbers to be sent via Json, Replace NaN values with None in SQAProfilerInterface
Replace NaN values with None in the SQAProfilerInterface class to maintain database parity. NaN values will be cast to null in OpenMetadata. This change ensures that data handling processes account for this conversion.
* fix: histogram overflow error
* test: Add Unit Test for Null and Null Ratio Metric
* chore: Address comments
* chore: Address comments
* fix: checkstyle and message
* fix: failing tests as null count works as expected
* mysql integration tests
* fix(data-quality): accept between with no bounds
add between filters only when the bounds are defined. if they are not (ie: resolve to 'inf' values), do not add any filters
* format
* consolidated ingestion_config
* format
* fixed handling of date and time columns
* fixed tests
* Add domain field to entity jsons
* add subdomains ui
* Add Domain To create Request
* Typo fix
* commit progress
* allow multiple domains
* - add userDomains and teamDomains
* - Restrict Api Access
* allow multiple domain support
* - Add Listing Restricition
- Add Search Restrictions
* - Field Fix
* - Inherit Team Domains
* remove glossary filter from ui
* - Fix Tag and User Filter
* - Id -> FQN in create
* - Missing domain on indexes
* add domain in creation of team, user and glossary
* fix merge conflicts
* fix feedbacks
* fix tests
* add domain playwright
* minor changes
* fix domain fetching calls
* Add Thread Listing with EntityDomain
* Fixes for Team Join
* Mkae list mutable
* Add missing import
* Fix Team Test
* User Test case fix
* Fix User Domain
* Minor bug fixes related to sub domains
* Fix data retaining in add sub domain modal after closing
* fix NavBar unit test
* Make Domain Access Work Via Policies
* Fix errors introduced after conflict resolutions
* update render function in SubDomainsTable.component.tsx to accept an array of owners instead of a single owner
* fix java checkstyle
* fix py-test
* format
* Fix the domain e2e owner field type
* Fix Failing Tests
* fixed pytest list index
* update userDomains and teamDomains to domains
* Entity Domain to Domain for thread and changeEvent
* FIx fields
* @Ovveride
* FIx
* Remove Duplicate Field
* Remove Duplicate Field
* Fix Failing Test
* fix domain related changes
* fix subdomains loader
* Fix Owners Order
* Enforce Domain Permissions
* fix domain cleanup
* Fix Test
* rever owner change
---------
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* Add multiple owners
* Multi Ownership
* Issue #17012: Multi User/Team Ownership
* Issue #17012: Multi User/Team Ownership
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 1
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 2
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 3
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 4
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 5
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 6
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 7
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 8
* Add Migrations for Owner Thread
* update ingestion for multi owner
* fix pytests
* fixed checkstyle
* Add Alert Name to Publishers (#17108)
* Add Alert Name to Publishers
* Fix Test
* Add Bound to Setuptools (#17105)
* Minor: fixed testSummaryGraph issue (#17115)
* feat: updated multi pipeline ui as per new mock (#17106)
* feat: updated multi pipeline ui as per new mock
* translation sync
* fixed failing unit test
* fixed playwright test
* fixed viewService click issue
* sorted pipeline based on test case length
* Added domo federated dataset support (#17061)
* fix usernames (#17122)
* Doc: Updated Doris & Redshift Docs (#17123)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
* Fix#12677: Added Synapse Connector - docs and side docs (#17041)
* Fix#17098: Fixed case sensitive partition column name in Bigquery (#17104)
* Fixed case sensitive partiion col name bigquery
* update test
* #13876: change placement of comment and close button in task approval workflow (#17044)
* change placment of comment and close button in task approval workflow
* minor change
* playwright test for the close and comment function
* supported ref in activityFeedEditor
* fix playwright test
* added playwright test for data steward
* fix the test for the data streward user
* fix the close button not showing if task has no suggestions and icon fixes
* fix sonar issue
* change glossary and add suggestion button to dropdown button
* fix the glossary failure due to button change
* icon change for add tag and description
* fix glossary cypress failure due to button chnages
* changes as per comments
* MINOR: docs links fix (#17125)
* alation link fix
* dbt yaml config source link fix
* bigquery doc fix
* Explore tree feedbacks (#17078)
* fix explore design
* update switcher icon
* show menu when search query exists
* fix selection of active service
* fix type error
* fix tests
* fix tests
* fix tests
* MINOR: Databricks view TableType fix (#17124)
* Minor: fixed AUT test (#17128)
* Fix#16692: Override Lineage Support for View & Dashboard Lineage (#17064)
* #17065: fix the tags not rendering in selector after selection in edit tags task (#17107)
* fix the tags not rendering in selector after selection in edit tags taks
* added playwright test
* minor changes
* minor fix
* fix the tags not updating in edit and accept tag
* fix explore type changes for collate (#17131)
* MINOR: changed log level to debug (#17126)
* changed log level to debug
* fixed type
* changed type to optional
* Get feed and count data of soft deleted user (#17135)
* Doc: Adding OIDC Docs (#17139)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
* Doc: Updating Profiler Workflow Docs URL (#17140)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
* fix playwright and cypress (#17138)
* Minor: fixed edit modal issue for sql test case (#17132)
* Minor: fixed edit modal issue for sql test case
* fixed test
* Minor: Added whats new content for 1.4.6 release (#17148)
* MINOR [GEN-799]: add option to disable manual trigger using scheduleType (#17031)
* fix: raise for triggering system app
* added scheduleType ScheduledOrManual
* minor: remove "service" field from required properties in createAPIEndpoint schema (#17147)
* initial commit multi ownership
* update glossary and other entities
* update owners
* fix version pages
* fix tests
* Update entity_extension to move owner to array (#17200)
* fix tests
* fix api page errors
* fix owner label design
* locales
* fix owners in elastic search source
* fix types
* fix tests
* fix tests
* Updated CustomMetric owner to entityReferenceList. (#17211)
* Fix owners field in search mappings
* fix search aggregates
* fix inherited label
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 9
* Fix QUeries
* Fix Mysql Queries
* Typo
* fix tests
* fix tests
* fix tests
* fix advanced search constants
* fix service ingestion tests
* fix tests
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: k.nakagaki <141020064+nakaken-churadata@users.noreply.github.com>
Co-authored-by: Prajwal214 <167504578+Prajwal214@users.noreply.github.com>
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: harshsoni2024 <64592571+harshsoni2024@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
* Refactor output_handlers to a WorkflowOutputHandler class
* Add old methods as deprecated to avoid breaking changes
* Extract WorkflowInitErrorHandler from workflow_output_handler
* Fix static checks
* Fix tests
* Fix tests
* Update code based on comments from PR
* Update comment
* tests: refactor
refactor tests and consolidate common functionality in integrations.conftest
this enables writing tests more concisely.
demonstrated with postgres and mssql.
will migrate more
* format
* removed helpers
* changed scope of fictures
* changed scope of fixtures
* added profiler test for mssql
* fixed import in data_quality test
* json safe serialization
* format
* set MARS_Connection
* fix(data-quality): empty test suite
do not raise for empty test suite
* format
* dont need to check length in _get_test_cases_from_test_suite
* fix
* added warning if no test cases are found
* tests: refactor
refactor tests and consolidate common functionality in integrations.conftest
this enables writing tests more concisely.
demonstrated with postgres and mssql.
will migrate more
* format
* removed helpers
* changed scope of fictures
* changed scope of fixtures
* added profiler test for mssql
* fixed import in data_quality test
* json safe serialization
* format
* set MARS_Connection
* use SerializableTableData instead of TableData
* deleted file test_postgres.py
* fixed tests
* added more test cases
* format
* changed name test_models.py
* removed the logic for serializing table data
* wip
* changed mapping in common type map
* changed mapping in common type map
* reverted TableData imports
* reverted TableData imports
* reverted TableData imports
* feat(table-diff): added column validation
added column validation for table diff that will be carried out before running the row level diff. If a diff for the column exists, it will short circuit the test and report.
* fixed unit tests
* format
* - resolve column types more robustly
- changed test result metric to include "rows" or "columns"
* fix(data-quality): incompatible columns
gracefully fail when a column of incompatible type is submitted for a test case
* format
* added condition to handle only colum test cases
* fixed tests
* format
