* Fix#12964: filter draft dashboards from config
* Fix#12964: add unit test for qlik_sense
* Fix#12964: added UI and doc code
* Fix#12964: move includedraftdashboard flag from source_connection to source_config
* Fix#12964: filter draft dashboards in qlikcloud
* Fix#12964: add unit test for qlik cloud
* Fix#12964: remove unnecessary comments, code clean
* Fix#12964: pylint changes
* Implement Initial MultiThread suggestion
* Update all the ingestion sources to use the new ContextManager
* Fix missing wraps on decorator
* Fix Unittests
* Fix linters
* Fix linters
* Fix BigQuery UnitTests
* Add UnitTests to the newly created code
* Fix unittest
* change the threads from table to schemas
* Update README.md
* Small change suggested by Sonar
* Slight change to test a different way to multithread over tables
* Debug changes
* More multithread tests
* Remove uneeded wait time
* Testing
* refactor code based on removal of time.sleep
* Fix wrong paste
* Improve ExecutionTimeContextManager
* Fix missing .get() and unit tests
* Fix conflicting changes
* Update Multithread logic with the incremental extraction
* Fix linters
* Fix unittest
* Remove commented code
* Fix Unittests
* Fix checkstyle
* Change default to threads = 1
* feat(connector): add BigTable
* bigtable work
1. docstrings
2. tests
3. created a Row BaseModel
4. implemented a ClassConverter
* docs moved to separate PR
* format files
* minor cosmetic
- removed TODO
- changed headers' year to 2024 for new files
- fixed typos
* format
* formatting and comments
1. added missing docstrings.
2. abstracted the _find_instance method.
3. aliased the IDs used in the BigTable connection
* added comment regarding private key
* added comments regarding column families
* enclose get_schema_name_list in `try/except/else`
* format
* streamlined get_schema_name_list to include all logic in the try block
* linting: fix python linting
* fix: get column types from parquet schema for parquet files
* style: python linting
* fix: remove displayType check in test as variation depending on OS
* Create the iceberg connection schema
* Link the IcebergConnection configuration with the forms on the UI
* Add the pyiceberg dependency on the ingestion package
* Create the get_connection and test_connection functions
* First iteration on the iceberg ingestion logic
* Add A more comprehensive implementation of the Iceberg Source
* Add UnitTests
* Update icebergConnection definition
* Update the iceberg souce code based on new schema
* Updated icebergConnecgtion schema for simplicity and to be able to configure Converters
* Updated setup dependencies to be more flexible
* Updated get_owner_ref logic
* Fix formatting
* Changed the icebergConnection json schema structure to enable the ClassConverters
* Add the IcebergCatalog and IcebergFileSystem ClassConverters
* Refactor the code to take into account the new jsonSchema structure
* Fix formatting
* Add Documentation for the Iceberg Connector
* Fix Menu order for Iceberg
* ui: add Iceberg service icon and constant
* Fix DynamoDb Catalog issue due to how PyIceberg instantes it
* Changed uri title to URI
* Fix ClassConverter for Iceberg
* Fix GetSecretValue for password types
* Fix formatting
* Fix formatting
* Add Iceberg Connector Images for the docs
* Add pylint disable for Hacky super() call
* Add Iceberg.md for the UI docs
* Fix pylint complaint
* Fix pylint complaint
* Fix UnitTests
* fix type error and unit tests
* update pipeline type checks
* Fix Sonar Cloud complaints
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Changed for database
* Added changes for dashboard_service
* Changed for messaging service
* Changed for mlmodel service
* Changed for pipeline service
* Changed for search service
* Changed for objectstore service
* fixed wrong import
* fixed lint
* fixes
* fixed pytests
* fixed domo db pytest
* Fixed review comments
* global manifest option for storage services
* added a no metadata config source option for global manifest s3 services option
* merge fixes
* more merge fixes.
* black stuff
* test fixes
* formatting
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
