Onkar Ravgan
e21f38c5fd
handled null upstreamcol values ( #11835 )
2023-06-02 16:55:28 +05:30
Teddy
d0cffdcd66
Fixes Issue #11438 - Implement threshold and startegy for custom SQL ( #11847 )
...
* feat: Add threshold and strategy logic on the custom SQL object test
* feat: ran python linting
* feat: added safety checks for custom sql query
* feat: ran python linting
2023-06-02 09:41:31 +02:00
Teddy
c98a15ca19
Fixes #11705 - Update ingestion and backend to match new DQ flow ( #11836 )
...
* feat: refactor ingestion flow logic
* feat: ran python linting
* feat: update tests to match new workflow
* feat: ran python linting
* feat: update sample data test suite name
* feat: Added backend logic to support logical and executable test suites
* feat: clean up java and json code
* feat: added sample data for logical and executable test suites
* feat: remove executable from CreateTestSuite
* feat: ran python and java linting
* feat: added README info for data quality structure
* skipping cypress to keep main green
* fixed typescript type issue
---------
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2023-06-01 23:19:13 -07:00
Pere Miquel Brull
fdeea71671
Fix Looker explore git link & Add BitBucket reader ( #11837 )
...
* Add looker test connection step
* Add looker test connection step
* Update Credentials
* Fix explore link and add bitbucket reader
* Format
* Fix test
* Fix spline linting
* Fix import
2023-06-02 07:19:32 +02:00
Mayur Singal
b7a1066f5b
Fix #11781 : Fix atlas lineage issue ( #11832 )
2023-06-01 12:02:43 -07:00
Pere Miquel Brull
0a8b28ab7e
Fix redshift e2e with new lineage info ( #11843 )
...
* Fix redshift e2e
* Format
* Format
2023-06-01 09:44:27 +02:00
Ayush Shah
8e57b165a6
Fix LineTooLong ( #11833 )
...
* Fix LineTooLong
* Add Timeout ( fix lint )
2023-06-01 09:40:18 +05:30
Pere Miquel Brull
3966238703
Remove e2e system metrics test ( #11838 )
2023-05-31 18:02:17 +02:00
Pere Miquel Brull
11c07ee8ab
Fix #11516 - SAP Hana Connector ( #11777 )
...
* SAP Hana skeleton
* Add SAP Hana Connector
* Fix ingestion and docs
* Prep SAP Hana Profiler
* Linting
* Update index.md
* Revert: Update index.md
---------
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-31 16:00:31 +02:00
Mayur Singal
ae386d2e2b
Add Table Sample Data During Sample Data ingestion ( #11773 )
2023-05-31 12:23:45 +00:00
Mayur Singal
9e437fceec
Fix redshift view lineage parsing issue ( #11702 )
...
* Fix redshift view lineage parsing issue
* address review comments
2023-05-31 12:10:39 +00:00
Mayur Singal
b57bbf833f
Fix #11572 : Glue Support Partition Columns & Use Pydantic Models ( #11776 )
2023-05-31 12:03:34 +00:00
Pere Miquel Brull
158250353b
Fix hive e2e tests ( #11815 )
...
* Fix hive e2e tests
* Fix hive e2e tests
2023-05-30 17:15:55 +02:00
Pere Miquel Brull
a00fc7fef3
Increase wait time for system metrics e2e test ( #11804 )
2023-05-30 10:45:14 +02:00
Pere Miquel Brull
7ff00d399e
Install ibm-db-sa if not ARM ( #11798 )
2023-05-30 10:36:48 +05:30
Pere Miquel Brull
e02fc231da
Improve logging for test connection workflow ( #11794 )
2023-05-30 10:36:10 +05:30
Chirag Madlani
7adc291364
fix(ui): circular deps for entityReference.json ( #11760 )
...
* fix(ui): circular deps for entityReference.json
* Fix circular Dependency python
* Cap Delta Spark version
---------
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-26 18:02:21 +05:30
Pere Miquel Brull
5bd1b09402
Add new Looker test connection steps ( #11749 )
2023-05-24 18:01:08 +02:00
Mayur Singal
c44bff8fd3
Fix ValueError: too many values to unpack for col lineage ( #11723 )
2023-05-24 11:30:44 +05:30
Onkar Ravgan
3fbddc2a03
upgrade kafka dep ( #11721 )
2023-05-23 09:59:12 -07:00
Sriharsha Chintalapani
6509a3670a
Fix #11664 : Refactor patch_mixin to use jsonpatch lib ( #11696 )
...
* Fix #11664 : Refactor patch_mixin to use jsonpatch lib
* Migrate to jsonpatch
* Fix nested cols
* Format
* Update patch_description
* Table constraints
* tag
* owner
* column tag
* column desc
* Format
* Format
* Fix log
* Update dbt patch
* Update column fqn
* Fix test
* Fix tests
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-05-23 15:47:11 +02:00
Onkar Ravgan
7589b7eeef
fixed kafka basicaauth bug ( #11729 )
2023-05-23 19:05:01 +05:30
Onkar Ravgan
efb37fa7af
Fixed tableau e2e ( #11716 )
2023-05-23 15:36:08 +05:30
Pere Miquel Brull
1370325762
Fix typing_extension verson ( #11719 )
...
* fix extension
* fix extension
2023-05-23 10:09:31 +02:00
Onkar Ravgan
f677408f20
Fixes 11603: Tableau Data Sources as Data Models ( #11688 )
...
* added initial changes
* Added lineage details
* fixed pylint
* Fixed name
* Fixed empty lists
2023-05-22 18:07:02 +02:00
Teddy
9f2a02c718
fix: increase sleep delay for system tables ( #11704 )
2023-05-22 13:43:50 +00:00
Teddy
8c50d1af52
Fixes #4565 - Fetch Metrics from System tables ( #11645 )
...
* feat: fetch metrics from system tables
* feat: add permission doc for fetching metrics from system tables
* feat: fix E2E tests to reflect full table row count after table metric update
* feat: ran linting
* feat: fix doc string engine name + function typing
* feat: ran python linting
2023-05-22 09:04:18 +02:00
Teddy
ddbc7fe14d
Fixes #11570 - Add support for BQ Multi-project Profiler ( #11692 )
...
* fix: extracted profiler object from workflow and implemented factory to allow service base logic
* fix: ran python linting
* fix: renamed `base` to `base_profiler_source`
* fix: add logic to set correct database for BQ multi project ID connections
* fix: ran python linting
2023-05-20 14:22:53 -07:00
Mayur Singal
8ef98076ae
Fix view lineage not generated for oracle ( #11648 )
2023-05-19 10:47:55 -07:00
Pere Miquel Brull
0eb2201f94
Restructure NER Scanner internals ( #11690 )
...
* Simplify col name scanner
* Restructure NER Scanner internals
2023-05-19 18:21:01 +02:00
Mayur Singal
fafbfdaeab
Fix postgres view lineage parsing issue ( #11689 )
2023-05-19 17:47:50 +02:00
Ayush Shah
ad7258e7be
Fixes 10949: return Chunks for file formats & Centralize logic for different auth configs ( #11639 )
...
* Centralize Auth and File formats datalake
2023-05-19 18:54:28 +05:30
Pere Miquel Brull
d52d773707
Send encrypted automation workflow ( #11681 )
2023-05-19 15:04:42 +02:00
Mayur Singal
e9992a52a8
Fix #1604 : Add Spline Pipeline Connector ( #11562 )
...
* Fix #1604 : Add Spline Connector
* Add tests & grammer validation
* Spline UI Changes & Docs
* fix pipeline workflow doc
* chore: use common field for dbService name
* chore: use const for beta services
* chore: add service icon
* Update ingestion/src/metadata/ingestion/source/pipeline/spline/metadata.py
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-05-19 14:46:32 +05:30
Pere Miquel Brull
50ad38ea0f
Fix #11548 - Secrets Managers comms with OMeta ( #11602 )
...
* Remove secretsManagerCredentials from backend
* Remove secretsManagerCredentials from backend
* Add secrets manager loader
* Load SM in the ometa client
* Fix tests
* Fix tests
* Fix Lint
* Mock AWS region
---------
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-19 09:43:11 +02:00
Pere Miquel Brull
4626363fd8
Fix parsing for Storage ( #11663 )
2023-05-19 09:36:44 +02:00
Pere Miquel Brull
b60f198674
Fix GitHub reader headers and add docs ( #11675 )
...
* Fix header and add docs
* Format
2023-05-19 09:36:30 +02:00
Chirag Madlani
763b96b8fe
bump(ui): rjsf to v5 ( #11549 )
...
* bump(ui): rjsf to v5
It will fix validation and extra parameter issues
* address comments
fix code smells
fix cypress failure
* revert schema files
* bigquery fix
sonar cloud fix
* fix big query
* fix superset
* fix cypress
* fix cypress for big query
* fix failures
* fix java issue
* revert connectionBasicType.json
create custom SupersetConnection.json
* fix Tableau service issues #11637
2023-05-18 08:03:54 -07:00
Pere Miquel Brull
8795337f88
Clean NER Scanner imports ( #11653 )
2023-05-18 12:53:22 +02:00
Mayur Singal
e4997c3749
Fix #11571 : Support custom database name for glue ( #11631 )
2023-05-18 14:16:56 +05:30
Onkar Ravgan
e6a7ffc57c
Fixes: 11540 Added api_version and domain fields to Salesforce ( #11618 )
...
* Added sales new fields
* fixed image after merge
* Fix Failing Test
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-05-18 10:53:44 +05:30
Pere Miquel Brull
1b90badd0e
Restructure PII processor ( #11640 )
...
* Restructure PII processor
* Restructure PII processor
* Format
2023-05-17 15:58:17 +02:00
Mayur Singal
ea615eed9d
Fix elasticsearch error ( #11635 )
2023-05-17 15:48:20 +05:30
Pere Miquel Brull
c0ea11a477
Add libsasl deps ( #11613 )
2023-05-16 14:32:48 +02:00
Mayur Singal
b53a362772
Increase delay for system metric in e2e tests ( #11607 )
2023-05-16 06:57:36 +00:00
Pere Miquel Brull
019014b8d3
11579 Task - Add () in sample data ( #11591 )
2023-05-15 10:09:57 +02:00
Onkar Ravgan
3d9d4416b7
Fixed incompatible column name for Postgres version 11.6 ( #11536 )
...
* postgres col name on version
* Added dependancy
* Added paranthesis validation
* review comments and tests
2023-05-15 11:48:03 +05:30
Akash Jain
92d5bfa94e
fix: versions in main branch ( #11478 )
...
* fix: versions in main branch
* Prepare main branch for next release 1.1.0
* prepare main for latest release 1.0.1
2023-05-15 10:42:29 +05:30
Mayur Singal
e61bb3cf0d
Fix snowflake E2E: add system metric sleep time ( #11569 )
2023-05-12 14:09:57 +05:30
Onkar Ravgan
cff403a05a
Validate if tags are created before attaching them to CreateRequest ( #11554 )
...
* Added tags validation
* typo fixed
2023-05-11 16:04:55 +00:00
