* fix: added code for separate engine and session for each project in rofiler and classification and refactor billing project approach
* fix: added entity.database check, bigquery sampling tests
* fix: system metrics logic when bigquery billing project is provided
* Initial Implementation of Adding Status and Reviewers to assets for workflows
* Update generated TypeScript types
* Copilot Review Comments Addressed
* Removed DataProduct Reviewer Inheritance as it is irrelevant
* Commit: Classification has status and reviewers, DataContract uses the same status enums, changed the logic to be APPROVED instead of Active, DataContract can have null status as seen in tests, Changed Workflow to use workflowStatus instead of status as it is contradicting with the approval status, Fixed Tests
* Default for reviewers is null
* Default for reviewers is createSchema
* Addressed CoPilots comments
* Update generated TypeScript types
* Workflow status to workflowStatus in db and migrations
* Revert "Workflow status to workflowStatus in db and migrations"
This reverts commit 676e8789358654bc6f980f855c372f33c22fc40b.
* Changed status to entityStatus in the schema files
* Java Implementation of Default Status, Search Client improvements and Test fixes and new tests
* Adding entityStatus and reviewers in the searchIndex mappings and common attributes
* Data Migration scripts to change the glossaryTerm and dataContract structure
* Update generated TypeScript types
* Fixed zh/spreadsheet index json error
* Fix Postgre migration script
* Changed the entityStatus.json to status.json
Removed the duplicates of entityStatus in the indexMapping
Modified the sample data to take in EntityStatus.Approved instead of ContractStatus.Active
* Update generated TypeScript types
* dummy commit
* Fix UI Build Issues with the New EntityStatus
Fix py tests
* Migrations for all the entities that need entityStatus
* Update generated TypeScript types
* Removed Post Migration scripts
* Fix UI and py for entityStatus
* Update generated TypeScript types
* Fix: DataContractResourceTest
* Fix UI and py for importing entityStatus
* UI to show and fetch Reviewers
* cleanup
* Removed Overridden SetDefaultStatus in GlossaryTermRepository
* Removed unnecessary validation
* Added entityStatus in search_entity_index_mapping.json
* Fixed DataContractResourceTest
* mvn spotless apply and fix migration scripts
* fix tests
* fix type error
* fix advanced search tests
* Status comparison using enums and supportsStatus to supportsEntityStatus
* mvn spotless apply
* fix merge conflict
* update entity status
* fix tests
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
* MINOR: Improve UDF Lineage Processing & Better Logging Time & MultiProcessing (#20848)
* Fix multiprocessing with better memory management and Airflow 2+ compatibility
* Add support for both multiprocessing and multithreading for relevant platforms
* Handle conflicting cross-db lineage changes of service_name parameter change
* Handle stored proc queries without caching all and increase the thread timeout times to cover 100% lineage
* Fix `get_table_query` inheritance and pylint
* Remove mocks from db_utils tests
* Better db_utils test and fix the service_names parameter in case of schema_fallback
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* Fix Oracle DataDiff and Change Oracle Connection to BaseConnection
* Add small unittest
* Fix Test
* Fix logic, to void other engines to denormalize table/schema names
* Add calculated view columns' formula parsing logic with correct source reference
* Handle top level column formula parsing and pass formula expression in column lineage detail
---------
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
* fix: ingestion fails for Iceberg tables with nested partition column
* test: added test to cover nested partition column for iceberg
* refactor: used if-else in tablePartition check
* fix: partition_column_name & column_partition_type typo
* Enhance Sample Data Generation: Update table and column limits, add description and owner fields to table creation requests in sample_data.py
* Refactor SampleDataSource: Improve readability by adjusting conditional formatting for owner checks in sample_data.py
* Reduced number of tables per schema to 10
* Update sample_data.py: Reduce the maximum number of columns per table from 2000 to 200 for improved data generation efficiency
* Refactor BigQuery client type hinting in bigquery_utils.py
- Updated type hint for the BigQuery client to use forward declaration for better compatibility with type checking.
- Moved import statement for google.cloud.bigquery inside TYPE_CHECKING block to optimize imports during runtime.
* Refactor BigQuery client import structure in bigquery_utils.py
- Moved the import statement for google.cloud.bigquery inside the TYPE_CHECKING block to enhance type hinting compatibility.
- Adjusted the import location for better runtime performance and adherence to best practices.
