* Deprecate location and old storage service
* Format
* Fix test
* Refactor
* Clean location
* Rename object store to storage
* Rename object store to storage
* Rename object store to storage
* Format
* Format
* Refactor object store for storage
* Refactor object store for storage
* Rename object store to storage
* Fix test
* Fix test
* Format
* chore(ui): change Objectstore to Storage
* Fixes
* Fix test
* Remove storage service from Glue cypress
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Organise calls
* Prepare skeleton
* Add looker model handling
* Parse files as sql
* Handle labels
* Linting
* Format
* Fix version
* Also check the API for explore lineage
* fix: logic for test suite config workflow
* fix: added caching for system metrics (snflk and bq)
* fix: linting
* fix: added tearDown logic for tests suite/case
* fix: raise more informative error message when service is not found
* fix: profiling for struct table
* fix: linting
* fix: added tests for struct and nestedt struct for get_columns
* Added mark delete logic
* Final test and optimization
* After merge fixes
* Added include tags for dash pipelines dbt
* added docs and fixed test
* Fixed py tests
* Added UI changes for following newly added fields:
- markDeletedDashboards
- markDeletedMlModels
- markDeletedPipelines
- markDeletedTopics
- includeTags
* Fixed failing unit tests
* updated json files of localization for other languages
* Improved localization changes
* added localization changes for other languages
* Updated mark deleted desc
* updated the ingestion fields descriptions in the ingestion form for UI
* automated localization changes for other languages
* updated descriptions for includeTags field for dbtPipeline and databaseServiceMetadataPipeline json
* fixed issue where includeTags field was being sent in the dbtConfigSource
* Added flow to input taxonomy while adding BigQuery service.
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Prepare the new test connection ingestion logic
* Update test assert
* Update Test Connection for SQA Sources
* Correct return type and method doc
* Handle decryption
* Non SQA Database Sources
* Add the run_automation script in ingestion-base
* Dashboard Test Connection Changes
* Pipeline, Messagin, MlModel & Metadata Sources
* ui: test connect flow-1
* Unmask connection parameters before sending to Ariflow
* ui: test connect flow-2
* Address review comments and pylint
* pytest fix
* ui: test connect flow-3 (refactoring and style fix)
* ui: test connect flow-4 (fix test connection status logic)
* sync local file
* ui: test connect flow-5 (fix lowercase issue and styling)
* ui: test connect flow-5 (show toast notifications)
* test: add unit test
* ui: test connect flow-5 (update service page test connection button)
* Databrick fix & pytest fix
* pylint
* Update test
* Fix merge
* S3 Test connection
* add style for mandatory step
* sync locales
* chore: add service name in workflow request
* Unmask using original service connection parameters
* Fix test connection unmasking
* Wrap inspector function to eliminate error outside test conn
* Fix linting
* fix:cy test
* Fix linting
* address comment
* refactor and fix connection type casing issue
---------
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* Prep skeleton for object store connector
* First of of ingestion piepline for bucket containers.
* addressed PR review comments.
* swaped random with secrets due to security warning.
* linter issues.
* added prefix population for containers.
* more linting
* removed temp file.
* object store unit tests, PR comments and minor refactorings
* docs update and new TestConnection approach.
* linting
* removed isStructured field from containermeta schema
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Change how status is handled after running workflow
* Reset changes in config files
* Add auxiliary Summary class
* Improve failures handling
* Pylint error
* Pylint error
* Show result in table
* Add test
* Fix setup.py
* Add comments
* fix: chart url
* fix: chart url
* changes as per comment
* added test and pydantic for chart url
* changed as per comment
* changed as per comment
* fix python test
* changes as per comment
* fix python test
* added query as an entity
* changed name of the variables and methods
* Added Resource Descriptors
* testcase bug fix
* addressing comments
* added script for table query migration
* added script for table query migration postgresql
* bug fix
* db change for script test
* added current timestamp
* change db config from postgresql to mysql
* added extension to use fucntion gen_random_uuid()
* solving maven ci
* added queryUsage and change is migration script
* addressing comments
* addressing comments
* added queryUsage relation and testcase
* added api to insert queries in bulk
* .
* fix a test case which was failing due to latest changes
* Ingestion Changes for Query as Entity
* move query changes to latest sqls
* added tags and owner
* update PR for Query as Entity
* update type
* fixed pagination
* fix path param
* fix TestCases
* add validation criteria
* removed exisitng query apis
* checkstyle fix
* remove vote from put
* remove vote from put
* Query As Entity Ingestion Changes
* Remove unused func
* update Review Comments
* update Review Comments
* remove previous changes for Query and Update Tests
* moved Checksum to Query Util Class
* update python api
* fix python checkstyle
* Fixed Tests
* Fix pytest
* remove space changes
* remove space changes
* Fixed put_addFollowerDeleteEntity_200
* Fix usage ingestion
* Update Python SDK and tests
* pylint fix
---------
Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* added flag for overriding owners
* docs update
* adding owners in patch call
* adding owners in patch call
* fix: python test case
* changes as per comment
* changes as per comment
* ui changes
* Update sqllineage-openmetadata version + add timeout
* Pyimpala fix colnames, comments and dialect sql compilation (#10470)
* Fix col names and comments for impala hive
* Fix cols, comments and impala sql compilation
* Handle hive types
* Format
* Added doc in avro array and tests (#10473)
* Fixed: Add job definition id field for dbt cloud in UI #10269 (#10472)
* fixed Add job definition id field for dbt cloud in UI #10269
* sync-localization file
* fixed failing unit test and add unit test for the changes
* Address PR comments
* Update tests
* Pylint clean
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
* Fix#10429: Kafka Sample data improvements and adding support for JSONSchema and Protobuf
* Fix#10429: Kafka Sample data improvements and adding support for JSONSchema and Protobuf
* Fix#10429: Kafka Sample data improvements and adding support for JSONSchema and Protobuf
* Fix#10429: Kafka Sample data improvements and adding support for JSONSchema and Protobuf
* Added top level parsing and unit tests
* fix(ui): show schemaText and fields both
* fix no data placeholder for fields & schema text
* addressing comments
* fixed py checkstyle
---------
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* Mask sensitive info from API responses
* Rename converter classes
* Add missing Java classes from JSON schemas and class converters
* Update test service connection schema
* Update datalakeConnection JSON schema and fix some tests
* Fix AlertsRuleEvaluatorResourceTest and minor error in run_local_docker.sh
* Fix Pipeline and Database service tests
* Minor refactor
* Fix CsvUtilTest
* Fix EventMonitorFactoryTest
* Fix CloudWatchEventMonitorTest
* Update datalake metadata
* Update bigquery metadata
* Fix test connection functionality
* Fix OMeta service api test
* Update gcsValues title and revert changes in GH actions
* Mask sensitive enabled by default for local docker
* Add missing tests
* Address PR comments
* Address PR comments
* fix ui breaks on gcsValues.json
* Address PR comments
* Minor refactor
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* WIP: pii tagging using spacy
* added test cases and changes as per comment
* fix python checkstyle
* fix python checkstyle
* added score, test_cases and docs update
* solved merge conflict
* fix python checkstyle
* remove pii tagging using regex
* fix python test
* lib changes and added some test case
* changed as per comment
* fix: python test
* fix: changes to get source_config
* fix: changes as per comment
* Update test connection
* Update test connection
* Update test connection with Tags
* Change based on comments
* Fix Pytest
* Service connectin arg in all connecter
* Fix pytest and conflict
* feat(profiler): renamed module to
* feat(profiler): added dbt-artifacts-parser to test setup.py
* feat(profiler): refactor workflow and interface
* feat(profiler): linting
* feat(profiler): removed old profiler modules
* feat(profiler): added support for value and integer range partition
* feat(profiler): fixed linting
* feat(profiler): added partitionning support for datalake profiler
* feat(profiler): removed `ProfilerInterfaceArgs` class
* feat(profiler): address comments
* feat(profiler): Added `OTHER` as an `IntervalType` for UI type generation
