* WIP - MINOR - Clean ingestion dependencies
* test
* test
* Clean imports
* add pyiceberg for test
* Revert "add pyiceberg for test"
This reverts commit ab26942736586f089a57a644ffd727aca200db62.
* add pyiceberg for test
* Remove docker dep
* clean local docker sh
* MINOR - AKS Airflow troubleshooting docs
* Fix action
* clean local docker sh
* Mantain the OperationType Order when considering the dividing groups
* Remove reordering the jsonpatch operations from the backend
* Fix checkstyle
* Fix UnitTests to comply with no reordering
* Initial idea on how to fix our current jsonpatch builder from python
* fix(JsonUtils): Change JSONPatch library used
When creating a JSONPatch by using the 'createDiff' method, the library
we are using is not returning a correct JSONPatch when removing multiple
items from an array.
Since the library doesn't provide good ways to override this behavior
and fix it, we decided to move away from it and use the json-patch
library only for this specific operation.
* Fix linters
* Add docstrings
* Refactor patch updated on ingestion framework
* Add UnitTests
* Fix linters
* fix: update antlr regex for entity fqn
* fix: update antlr rule to allow single character
* style: ran python linting
* fix: updated antlr token for NAME_OR_FQN
* Fix#12964: filter draft dashboards from config
* Fix#12964: add unit test for qlik_sense
* Fix#12964: added UI and doc code
* Fix#12964: move includedraftdashboard flag from source_connection to source_config
* Fix#12964: filter draft dashboards in qlikcloud
* Fix#12964: add unit test for qlik cloud
* Fix#12964: remove unnecessary comments, code clean
* Fix#12964: pylint changes
* Implement Initial MultiThread suggestion
* Update all the ingestion sources to use the new ContextManager
* Fix missing wraps on decorator
* Fix Unittests
* Fix linters
* Fix linters
* Fix BigQuery UnitTests
* Add UnitTests to the newly created code
* Fix unittest
* change the threads from table to schemas
* Update README.md
* Small change suggested by Sonar
* Slight change to test a different way to multithread over tables
* Debug changes
* More multithread tests
* Remove uneeded wait time
* Testing
* refactor code based on removal of time.sleep
* Fix wrong paste
* Improve ExecutionTimeContextManager
* Fix missing .get() and unit tests
* Fix conflicting changes
* Update Multithread logic with the incremental extraction
* Fix linters
* Fix unittest
* Remove commented code
* Fix Unittests
* Fix checkstyle
* Change default to threads = 1
* Fix#11868: Duplicate query should throw an error of entityExists
* Fix#11868: Duplicate query should throw an error of entityExists
* fix test
* fix test
* Fix uniquee constraint for checksum in Postgres
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* fix: add cli support for computePassedFailedRowCount
* fix: div zero error and improve empty table message
* doc: updated test case page
* style: ran python linting
* Add Custom Propety Config to store format, enum values, entity types
* Fix import statements and remove unused code
* Add Custom Propety Config to store format, enum values, entity types
* Add support for enum field type in custom properties
* update name in customPropertyConfigTypeValueField
* add custom property config column in custom property table
* Update padding-left in block-editor.less
* Add enum value translation for multiple languages
* update placeholder of config
* fixed python sdk
* add enum type in property value
* add unit tests
* Add Custom Propety Config to store format, enum values, entity types
* update ui to handle the enum config and validation
* Fix enum value handling in EditCustomPropertyModal and PropertyValue
* Update CustomProperty.md with enum values and multi-select option
* add cypress test
* add cypress for multiselect enum value
* Add tests for enum props
* add cypress for editing the enum property
* Add validations to enum
* Fix dependency issue
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
* feat(nosql-profiler): row count
1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.
* feat(nosql-profiler): row count
1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.
* added profiler support for mongodb connection
* doc
* use int_admin_ometa in test setup
* - fixed linting issue in gx
- removed unused inheritance
* moved the nosql function into the metric class
* formatting
* validate_compose: raise exception for bad status code.
* fixed import
* format
* feat(nosql-profiler): added sample data
1. Implemented the NoSQL sampler.
2. Some naming changes to the NoSQL adaptor to avoid fixing names with the profiler interface.
3. Tests.
* added default sample limit
* feat(nosql-profiler): row count
1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.
* added profiler support for mongodb connection
* doc
* use int_admin_ometa in test setup
* - fixed linting issue in gx
- removed unused inheritance
* moved the nosql function into the metric class
* formatting
* validate_compose: raise exception for bad status code.
* fixed import
* format
* feat(connector): add BigTable
* bigtable work
1. docstrings
2. tests
3. created a Row BaseModel
4. implemented a ClassConverter
* docs moved to separate PR
* format files
* minor cosmetic
- removed TODO
- changed headers' year to 2024 for new files
- fixed typos
* format
* formatting and comments
1. added missing docstrings.
2. abstracted the _find_instance method.
3. aliased the IDs used in the BigTable connection
* added comment regarding private key
* added comments regarding column families
* enclose get_schema_name_list in `try/except/else`
* format
* streamlined get_schema_name_list to include all logic in the try block
