20 Commits

Author SHA1 Message Date
Mohit Yadav
b3e0470882
Migration Fixes (#16131)
* - Migration Fixes

* - Migration Fixes

* - Typo fix

* - Make warn to error

* - Update Migration to show Query Status

* - Fix Query in POstgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-05-07 22:07:25 +05:30
Onkar Ravgan
0f0bccdd45
Converted and fixed pipelinestatus timestamps to milliseconds (#13670)
* fixed pipelinestatus timestamps in mills

* Added migrations
2023-10-20 09:39:24 -07:00
Pere Miquel Brull
8cf8720a9d
Clean Airflow Lineage Backend and migrate status to millis (#13666)
* Clean Airflow Lineage Backend and migrate status to millis

* Format

* chore(ui): update executions startTs and endTs to millis

* Remove lineage providers

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-10-20 15:42:38 +02:00
Pere Miquel Brull
255bfb95b1
Remove duplicates from entity_extension_time_series and add the const… (#13626)
* Remove duplicates from entity_extension_time_series and add the constraing if missing

* Add sort buffer and work mem

* Revert "Add sort buffer and work mem"

This reverts commit fcfff5feb60c9212bb7c1cad34b524dc8c03bfc5.
2023-10-19 12:15:02 +02:00
Teddy
e57849b732
Fixes #12298 - Update report data type to camel case (#13505)
* fix: updated DI to camelCase

* fix: ran linting

* fix: added migration

* fix: remove extra parenthesis in migration file

* fix: psql migration query

* fix: OS compose host

* fix: removed commented code block
2023-10-11 08:14:21 +02:00
Mohit Yadav
e577333b54
- Add Latest App Run Endpoint (#13511)
- Remove not required fields from app_extension table
2023-10-10 22:10:59 +05:30
Mohit Yadav
ab1a156b77
Build Applications Automations (#13318)
* [WIP] Applications

* Remove JobRunr Dependency

* Cleanup Files

* Reformatted

* Add Api to delete schedule for app

* Fix Applications issue

* added missing tables for application automation

* fixed buil failure

* Add Tables for Applications

* Migrate Search Reindexing Job to New Apps

* Fixed Merge Conflicts

* Fix Build

* Add DataInsightReportJob as App

* Review Comments Fix

* update err msgs

* Add Features fields

* Add Default Search Index Application

* Fix Failures

* Add Data Insights application

* Fix trigger and add update fields

* Add Stats context

* Fix Scheduler Initialization

* Fix Stats Issue

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: 07Himank <himank07mehta@gmail.com>
Co-authored-by: 07Himank <112613760+07Himank@users.noreply.github.com>
2023-10-10 11:32:37 +05:30
Mayur Singal
c0ababd8ad
Fix #13336: Clean Mark All Deleted Table Flag (#13344) 2023-10-06 16:04:54 +05:30
Sriharsha Chintalapani
762824665a
Fix #12655: Customizable Landing Page - Add Personas, DocStore to support KnowledgePanels and Pages (#13229)
* Fix #12655: Customizable Landing Page - Add Personas

* Add Knowledge Panels

* Add Doc Store to store KnowledgePanels and Pages

* Add Doc Store to store KnowledgePanels and Pages schemas

* Fix tests

* Add Multiple Persona support for a user

* add more elaborate tests for docstore

* Fix Page and KnowledgePanel required fields

* Fix Page and KnowledgePanel required fields

* Fix Page and KnowledgePanel required fields

* Fix build
2023-09-29 19:13:56 -07:00
Sriharsha Chintalapani
3a7f7486d0
Add indexes to name column for efficient ORDER BY name queries (#13383) 2023-09-29 14:19:49 +02:00
Pere Miquel Brull
18a4513ccc
Fix #13237 - Rename to instanceDomain and test DomoDashboard charts (#13247)
* Rename sandboxDomain to instanceDomain

* Test Get Charts in DomoDashboard

* Fix schemas

* Fix test

* Fix test

* Rename to Auto Tag PII

* Fix query test

* Fix query test

* Fix query test
2023-09-19 14:14:04 +02:00
Sriharsha Chintalapani
c2ed4f422f
Fix LifeCycle inconsistencies in Schema, make it common entity field (#13252)
* Fix LifeCycle inconsistencies in Schema; Add DELETE api

* set autocommit to true for non transactional

* make lifecycle common field for entities

* Add LifeCycle as common entity field

* Fix python life cycle code

* Fix search indexes

* remove unnecessary constant

* Add test back to entity resource test

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Add missing schema

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-09-19 14:03:57 +02:00
Pere Miquel Brull
22b0f44e38
Part of #12998 - Query Service & Lineage filter processed queries (#13215)
* Update mapping

* Prep

* prep

* Prep query migration

* Add query index deletion fix

* Docs and Maven CI fix

* Fix tests

* Add service filter

* Add query entity FQN col migration

* Fix lint

* supported serviceFQN in query api

* Prep repo

* Prep ES query search

* Do not recompute lineage

* Format

* Fix test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-09-19 07:37:47 +02:00
Pere Miquel Brull
f0995cbddc
Part of #12998 - Prep Stored Procedures Skeleton for Snowflake (#13121)
* Prep Stored Procedures Skeleton for Snowflake

* Update pylint and add migrations

* Fix test

* Reuse source url computation
2023-09-12 14:25:42 +02:00
Mohit Yadav
5f5aeef7f0
Transaction for Delete @UnitOfWork (#13115)
* Fix #12494: Jdbi Transactions opens a new connection per DAO causing recursive delete to not to rollback

* Fix #12494: Jdbi Transactions opens a new connection per DAO causing recursive delete to not to rollback

* rebase with main

* Fix styling

* Add jdbiUnitOfWork

* Improvements for Change Event Repository and missing Annotations

* Add connection commit/rollback

* push schemaChange fix

* Improvements for Change Event Repository and missing Annotations

* Pass wrapped collectionDAO everywhere

* Pass wrapped collectionDAO everywhere

* Separate event handlers transactions

* Fix checkstyle

* Wrap PUT , POST, and DELETE in transactions

* Add Patch for transactions

* Add primary key to tag.id

* Proxy internal @CreateSQLObjects

* Fix exception handling in ManagedHandleInvocationHandler

* Java typo

* Update schema files

* Checkstyle fix and conflicts resolve issue

* Remove @JdbiUnitOfWOrk from Feed Repository

* remove unnecessary @JdbiUnitOfWork annotation

* Test Failures fix

* Test Failures fix

* Increase wait time , changeEventDAO takes time due to handle.commit()

* commit change

* No need to use commit in ChangeEventHandler

* Add a lookup for avoiding transaction that are not started and should not be rolled back

* remove JdbiUnitOfWork.java

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-09-08 23:19:44 +05:30
Onkar Ravgan
d8e78a32d8
Added Life Cycle fields, API and ingestion integration (#13014)
* Added initial code

* temp commit

* Added snowflake changes

* Added postgres migration

* removed non req changes

* review comments one

* Added ometa tests

* lint fix

* fix lint naming

* fixed merge changes
2023-09-06 12:37:55 +02:00
Sriharsha Chintalapani
19b5c946a7
Fix #12167: Support for Stored Procedures as another entity under Database Schema (#12999)
* Add Stored Procedure Entity

* Stored Procedure repository

* Stored Procedure repository

* Fix #12998: Support for Stored Procedures as another entity under Database Schema

* Fix #12998: Support for Stored Procedures as another entity under Database Schema
2023-08-25 08:14:30 +02:00
Pere Miquel Brull
4862b0df28
Fix #10344 - Use ingestionPipeline retries when deploying (#12926) 2023-08-23 14:52:54 +05:30
Pere Miquel Brull
ccad601a4f
Fix postgres migration files (#12923) 2023-08-18 14:54:43 +02:00
Sriharsha Chintalapani
524e611edc
Deprecate Flyway and use Native migrations (#12865)
* Add ascii_bin for fqnhash, Add Constraints on timeseries

* Attemp of moving migrations to files

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Move upstream migrations to new path
2023-08-17 14:39:05 -07:00