* Restructured Reindexing + Added Automated Reindexing on Glossary import
* Revert Openmetadata yaml
* Fix pending issues
* Fix stats
* Refactor to workflows and SearchIndex
* Fix Failing tests
* Get status for stream and push status with
* fix UI build failing
* replace all with entity list before hit api
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* Use @Tag annotation to group APIs in the swagger documentation.
* Hide internal APIs
* Change API path events/subscription to events/subscriptions
* Change API path from automations/workflow to automations/workflows
* Change API path v1/testCase to v1/testCases
* Change API path v1/testDefinition to v1/testDefinitions
* Change API path v1/testSuite to v1/testSuites
* Rename Kpi and kpi in the documentation to KPI
* Change API path v1/testConnectionDefinition to v1/testConnectionDefinitions
* Update API section in the API documentation
* Fix test failures
* Correctly capitalize Test Cases and Test Suites in API docs
* chore(ui): add support for service documentation md file
* sync local
* chore: add method for fetching markdown file
* chore(ui): add support for service documentation
* chore: move fields to connections
* chore: update logic to fetch requirements
* chore: right panel component for service
* fix: key prop is not present in the skeleton component
* chore: only fetch md files when required fields are present
* chore: use hook for fetching airflow status
* chore: refactor add service component
* chore: remove id prefix and id separator prop from form builder
* fix: fieldName issue on right panel
* fix: active Field name issue
* fix:unit test
* test: add unit test
* chore: handle edit service form
* chore: add fallback logic
* fix: cy test
* chore: update service doc md files/folder structure,
* chore: push image example
* Athena docs
* Add glue docs
* Add hive related changes
* chore: take last field for fetching field doc
* add datalake
* Added connection information for oracle and redshift (english + french)
* fix: fallback logic
* Bigquery & Snowflake Requirements
* mysql and amundsen requirements (#10752)
* Revert removal of descriptions
* Add Doc For Mssql and Postgres
* Added powerbi conn md files
* Align requirements files
* Add Kafka and Redpanda
* refined powerbi docs
* Add Tableu requirements, move Athena and Glue fields, change footer some connectors
* Add missing connectors fields descriptions default
* re: datalake
* Add Tableau field descriptions
* fix: markdown styling
* chore: improve button styling
* chore: rename right panel to service right panel and move it to common
* fix: doc for select and list field , cy test.
* fix: unit test
* fix: test connection service type issue
* Added powerbi docs link in req
* Add info on hive
* Remove unused markdowns
* Add req for datalake
* add hive requirements header
* Snowflake & Biguqery
* Update Mssql and Postgres
* mysql and amundsen requirements updated
* Update Mssql and Postgres
* added username
* chore: fix cy expression issue
* chore: reset active field state on step change.
* fix: affix target container issue
* fix: unit test
* fix: cypress for postgres and glue
---------
Co-authored-by: Milan Bariya <52292922+MilanBariya@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: NiharDoshi99 <51595473+NiharDoshi99@users.noreply.github.com>
Co-authored-by: Milan Bariya <milanbariya12@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
* Added mark delete logic
* Final test and optimization
* After merge fixes
* Added include tags for dash pipelines dbt
* added docs and fixed test
* Fixed py tests
* Added UI changes for following newly added fields:
- markDeletedDashboards
- markDeletedMlModels
- markDeletedPipelines
- markDeletedTopics
- includeTags
* Fixed failing unit tests
* updated json files of localization for other languages
* Improved localization changes
* added localization changes for other languages
* Updated mark deleted desc
* updated the ingestion fields descriptions in the ingestion form for UI
* automated localization changes for other languages
* updated descriptions for includeTags field for dbtPipeline and databaseServiceMetadataPipeline json
* fixed issue where includeTags field was being sent in the dbtConfigSource
* Added flow to input taxonomy while adding BigQuery service.
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Handle test connection for S3 and new status
* Remove exception
* Update endpoint
* Update endpoint
* Update endpoint
* chore: remove toast messages for test connection
* chore: change objectstoreService to objectStoreService
* minor fix
* Use the default service param for listing filters
* Format
* Format
* address comments
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* improvement in pii tagging
* fix conflict and changes as per comment
* Added confidence field
* changes as per comments
* Apply suggestions from code review
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
* Add a configurationto skip Snowflake transient and tmp tables
* Fix Python checkstyle
* add separate query for transient tables
* Move skipTempTables into SnowflakeConnection
* Fix merge conflict
* change skip word to include
* Add title in json file
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Prepare the new test connection ingestion logic
* Update test assert
* Update Test Connection for SQA Sources
* Correct return type and method doc
* Handle decryption
* Non SQA Database Sources
* Add the run_automation script in ingestion-base
* Dashboard Test Connection Changes
* Pipeline, Messagin, MlModel & Metadata Sources
* ui: test connect flow-1
* Unmask connection parameters before sending to Ariflow
* ui: test connect flow-2
* Address review comments and pylint
* pytest fix
* ui: test connect flow-3 (refactoring and style fix)
* ui: test connect flow-4 (fix test connection status logic)
* sync local file
* ui: test connect flow-5 (fix lowercase issue and styling)
* ui: test connect flow-5 (show toast notifications)
* test: add unit test
* ui: test connect flow-5 (update service page test connection button)
* Databrick fix & pytest fix
* pylint
* Update test
* Fix merge
* S3 Test connection
* add style for mandatory step
* sync locales
* chore: add service name in workflow request
* Unmask using original service connection parameters
* Fix test connection unmasking
* Wrap inspector function to eliminate error outside test conn
* Fix linting
* fix:cy test
* Fix linting
* address comment
* refactor and fix connection type casing issue
---------
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* Prep skeleton for object store connector
* First of of ingestion piepline for bucket containers.
* addressed PR review comments.
* swaped random with secrets due to security warning.
* linter issues.
* added prefix population for containers.
* more linting
* removed temp file.
* object store unit tests, PR comments and minor refactorings
* docs update and new TestConnection approach.
* linting
* removed isStructured field from containermeta schema
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Rename Workflow package to automations
* Add list filter by root container & counts api
* Rename path
* Add glossary and glossaryTerm to count API
* Unify SqlQuery when there is no diff in mysql or psql
* chore: address feedback for object store service page
* Update glossary tests
* chore: add containers count on my data page
* fix: tag removal issue
* test: add unit test
* address comments
* address comment
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* added query as an entity
* changed name of the variables and methods
* Added Resource Descriptors
* testcase bug fix
* addressing comments
* added script for table query migration
* added script for table query migration postgresql
* bug fix
* db change for script test
* added current timestamp
* change db config from postgresql to mysql
* added extension to use fucntion gen_random_uuid()
* solving maven ci
* added queryUsage and change is migration script
* addressing comments
* addressing comments
* added queryUsage relation and testcase
* added api to insert queries in bulk
* .
* fix a test case which was failing due to latest changes
* Ingestion Changes for Query as Entity
* move query changes to latest sqls
* added tags and owner
* update PR for Query as Entity
* update type
* fixed pagination
* fix path param
* fix TestCases
* add validation criteria
* removed exisitng query apis
* checkstyle fix
* remove vote from put
* remove vote from put
* Query As Entity Ingestion Changes
* Remove unused func
* update Review Comments
* update Review Comments
* remove previous changes for Query and Update Tests
* moved Checksum to Query Util Class
* update python api
* fix python checkstyle
* Fixed Tests
* Fix pytest
* remove space changes
* remove space changes
* Fixed put_addFollowerDeleteEntity_200
* Fix usage ingestion
* Update Python SDK and tests
* pylint fix
---------
Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* added flag for overriding owners
* docs update
* adding owners in patch call
* adding owners in patch call
* fix: python test case
* changes as per comment
* changes as per comment
* ui changes
* Do not store OM connection
* Migration to remove the server connection
* Update tests
* Add workflow masking and secrets manager
* Fix failing test
---------
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
