* Search RBAC: process multiple view policies that can grant access to different set of entities
* Search RBAC: process multiple view policies that can grant access to different set of entities
* Search RBAC: process multiple view policies that can grant access to different set of entities
* Improve parallelization in SearchIndexApp
* Improve parallelization in SearchIndexApp
* add countdown latch
* typo
* Status issues - #1
* Improve search indexing
* cleanup the code
* schemas
* Status Updates Via WebSockets
* Add Updates
* Make Logs Debug
* show duration in table
* add searchIndexApp params to openmetadata-ops.sh reindex commandline
* Fix code style
* Add Parallel Entity Level Reader
* Make code more readable
* Send App Run Record instead of jobData
* Sned App Run Record WebSocket for Data Insights
* ui: use socket connection to get the real-time updates of application data
* Send WebSocket Final Update
* ui: remove failedRecords, totalRecords and successRecordsfrom entities record
* Invert Condition
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Add default ChangeEvent template and Slack destination test template
* fix conflict
* initial ingestionPipeline template
* clean templates.
* clean templates
* teams templates.
* add getBoldWithSpace()
* java check style
* google chat templates.
* Refactor GChat and Teams message for template data handling for DQ_Template_Section and General_Template_Section.
* Refactor slack message for template data handling for DQ_Template_Section and General_Template_Section.
* improvements gChatMessageDecorator
* improvements slackMessageDecorator
* improve MSTeamsMessageDecorator
* Call templates per entityType and refactor code
* Update EmailPublisher to use testEmail template instead of changeEvent template for sendTestMessage
* add parameterValues sections for teams
* Update SlackMessage to have attachments.
Update DQ template.
* Update dq templates, build dq template data in MessageDecorator
* remove IngestionPipeline template
* move SlackCallbackResource into a separate class. Fix tests.
* simulate timeout and 300.
* remove the validation of messages.
* fix teams MessageDecorator
* remove unused code from SlackMessageDecorator
* fix owners and tags data population in the template
* fix: changes in test case and test case result should load different templates.
* fix: list records from latest to earliest.
* refactor: remove listing unprocessed events from listEvents API.
* refactor: EventSubscriptionDiagnosticInfo.
* Add export apis back and new apis should be called exportAsync
* ui: use exportAsync endpoint for export assets
* fix build failure
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
* fix getSuccessfullySentChangeEventsForAlert
* migrations.
* offset-based pagination
* add startingOffset in EventSubscriptionDiagnosticInfo
* fix missing paginationOffset
* Record successfully sent change events in a separate table, clean up associated successful and failed records on alert deletion.
* Remove oldest 30 records if record count reaches 100 for successful_sent_change_events.
* list unprocessed changeEvents based on alert subscription rules, ensuring only relevant unprocessed events are listed.
* refactor: enhance eventSubscriptionDiagnosticInfo
Added new fields to the eventSubscriptionDiagnosticInfo schema to support detailed diagnostics:
- `relevantUnprocessedEventsCount`: Count of relevant unprocessed events based on alert filtering rules.
- `relevantUnprocessedEventsList`: List of relevant unprocessed events based on alert filtering rules.
- `totalUnprocessedEventsCount`: Count of all unprocessed events.
- `totalUnprocessedEventsList`: List of all unprocessed events.
* Refactor: delete 30 oldest records per alert when the count reaches 100 records
* fix: table creation issue in PostgreSQL
* fix tests.
* scheduleCleanupJob
* remove in-line deletion of records.
* fix postgres table structure issue.
* feat: added column value to be in expected location test
* fix: renamed value -> values
* doc: added 1.6 documentatio entry
* style: ran python linting
* fix: move data packaging to pyproject.yaml
* fix: add init file back for data package
* fix: failing test case
* Fix TestCaseResource testCaseInheritedFields
* Fix TableResourceTest
* Fix tests
* remove setInheritedFields in listing of search entities
* Fix tests
* remove unique entity name creations
* add specific names in search listing tests
* add large size for fetching from ES, as other tests adds to the table_search_index
* Fix tests
* fix tests
* Make Export CSV Async API, websocket to push data back
* ui: implement csvExportChannel to receive entity export data via websocket
* ui: update flow to show dialogbox with the response message and once data is available download the file
* minor: remove unused code
* ui: show inline banner for success and error message
* remove unused locale keys
* ui: clear existing csvexport job on sucess
* fix failing UserResourceTest
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
* Set max_analyzed_offset in HighlightBuilder to Fix Highlighting Errors for Large Fields
* fix java checkstyle
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* Fix#18332: Do not consolidate changes for Parent Entities
* Fix#18332: Do not consolidate changes for Parent Entities
* added no data placeholder when no tags there
* fix failing playwright test due to consolidation changes and added the missing services
* Fix tests
* Fix parent entity change notifications
* Fix tests
* Fix parent entity change notifications
* Fix tests
* Fix tests
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
* create EventSubscriptionDiagnosticInfo
* API for failedEvents
* migrations
* add index on source
* list all failedEvents optionally filtered by source
* refactor
* refactor
* getSuccessfullySentChangeEventsForAlert API
* increase defaultValue of limit to 100
* resolve conflicts
* listEvents API with 'failed', 'successful,' and 'unprocessed' query parameters
* fix description
* eventSource as enums
* refactor name.
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* fix: entity link accepted characters
* fix: match all characters but `<>|::`
* fix: remove unnecessary negative lookbehind
* fix the test case search not working on add ingestion page in case of special characters in FQN
* style: ran java linting
* fix: failing testst
* style: ran java linting
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Add ER diag APIs and sample data
* fix pylint
* formatting fixes2
* fixed es client return
* fixed os client return
* supported TableDetailPage tabs as classBase for supporting collate only tabs
* Added schema Apis
* change the base class to .ts and move the component in the util files
* beautify function arguments
* Added optimizations
* Ingestion changes
* svg dimension change
* supported class base tab in databaseSchema
* supported classBase action button in schema table name column
* added further keys data for constraint modal
* fix sonar issue
* remove old method to override edit action on column and shifted to DisplayNameModal for fields
* supported table right panel component to further extends on collate side
* minor fix around duplicate constraint
* added support to update table constraints and column constraints in the UI
* code optimization and minor fixes
* review comments and multi col fix
* added queryFilter option in NodeSuggestion and tableConstrainst to fetch and use only in service tables
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
* MINOR - Renaming MetaPilot to CollateAI
* MINOR - Renaming MetaPilot to CollateAI
* MINOR - Renaming MetaPilot to CollateAI
* MINOR - Renaming MetaPilot to CollateAI
* fix
* moving limits migrations for 1.5.9
