* fix: added code for separate engine and session for each project in rofiler and classification and refactor billing project approach
* fix: added entity.database check, bigquery sampling tests
* fix: system metrics logic when bigquery billing project is provided
* Initial Implementation of Adding Status and Reviewers to assets for workflows
* Update generated TypeScript types
* Copilot Review Comments Addressed
* Removed DataProduct Reviewer Inheritance as it is irrelevant
* Commit: Classification has status and reviewers, DataContract uses the same status enums, changed the logic to be APPROVED instead of Active, DataContract can have null status as seen in tests, Changed Workflow to use workflowStatus instead of status as it is contradicting with the approval status, Fixed Tests
* Default for reviewers is null
* Default for reviewers is createSchema
* Addressed CoPilots comments
* Update generated TypeScript types
* Workflow status to workflowStatus in db and migrations
* Revert "Workflow status to workflowStatus in db and migrations"
This reverts commit 676e8789358654bc6f980f855c372f33c22fc40b.
* Changed status to entityStatus in the schema files
* Java Implementation of Default Status, Search Client improvements and Test fixes and new tests
* Adding entityStatus and reviewers in the searchIndex mappings and common attributes
* Data Migration scripts to change the glossaryTerm and dataContract structure
* Update generated TypeScript types
* Fixed zh/spreadsheet index json error
* Fix Postgre migration script
* Changed the entityStatus.json to status.json
Removed the duplicates of entityStatus in the indexMapping
Modified the sample data to take in EntityStatus.Approved instead of ContractStatus.Active
* Update generated TypeScript types
* dummy commit
* Fix UI Build Issues with the New EntityStatus
Fix py tests
* Migrations for all the entities that need entityStatus
* Update generated TypeScript types
* Removed Post Migration scripts
* Fix UI and py for entityStatus
* Update generated TypeScript types
* Fix: DataContractResourceTest
* Fix UI and py for importing entityStatus
* UI to show and fetch Reviewers
* cleanup
* Removed Overridden SetDefaultStatus in GlossaryTermRepository
* Removed unnecessary validation
* Added entityStatus in search_entity_index_mapping.json
* Fixed DataContractResourceTest
* mvn spotless apply and fix migration scripts
* fix tests
* fix type error
* fix advanced search tests
* Status comparison using enums and supportsStatus to supportsEntityStatus
* mvn spotless apply
* fix merge conflict
* update entity status
* fix tests
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
* MINOR: Improve UDF Lineage Processing & Better Logging Time & MultiProcessing (#20848)
* Fix multiprocessing with better memory management and Airflow 2+ compatibility
* Add support for both multiprocessing and multithreading for relevant platforms
* Handle conflicting cross-db lineage changes of service_name parameter change
* Handle stored proc queries without caching all and increase the thread timeout times to cover 100% lineage
* Fix `get_table_query` inheritance and pylint
* Remove mocks from db_utils tests
* Better db_utils test and fix the service_names parameter in case of schema_fallback
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* Fix Oracle DataDiff and Change Oracle Connection to BaseConnection
* Add small unittest
* Fix Test
* Fix logic, to void other engines to denormalize table/schema names
* Add calculated view columns' formula parsing logic with correct source reference
* Handle top level column formula parsing and pass formula expression in column lineage detail
---------
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
* fix: ingestion fails for Iceberg tables with nested partition column
* test: added test to cover nested partition column for iceberg
* refactor: used if-else in tablePartition check
* fix: partition_column_name & column_partition_type typo
* Add lineage to Exasol connector
* Update test_connection to return TestConnectionResult
* Add exasol tests & dependencies to tests in setup.py
* Opensearch is required for testing, so add it there
* Modify metadata
* Update documentation for lineage
* Apply formatting changes to code
* Apply make py_format
* fix(dq): enable ''Column values to be in set'' test case for boolean columns
Add BOOLEAN to supportedDataTypes array in columnValuesToBeInSet.json
to allow boolean column validation with predefined allowed values.
This enables users to enforce strict true/false validation on boolean
columns directly at the column level, resolving issue #22099.
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Add tests to the new feature
* Add migrations and columnValuesToBeNotInSet
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Process temp table graph in stored procedure processor and add db/schema filtering on lineage
* Add tests for stored procedure lineage processing
* Fix tests and py_format
* Fix the filters and log stored proc query count info
