Ayush Shah
6c21da0d7b
Fixes 12224: Can't ingest binary sample data ( #12564 )
2023-07-25 12:33:26 +05:30
Onkar Ravgan
94166aeb84
fixed mssql comments ( #12546 )
2023-07-24 11:03:11 +05:30
Ayush Shah
30946ab0e9
Fix Cast and Col checks - Clickhouse ( #12556 )
...
* Fix Cast and Col checks
* fix lint
2023-07-22 12:53:24 +05:30
Ayush Shah
246bf15476
Add Clickhouse profiler fix ( #12531 )
2023-07-21 10:19:56 +05:30
mgorsk1
cd347299d7
Fixes 12439: Remove braces from approx_percentile method (Trino/Presto) in profiler ( #12440 )
...
* 🐛 Remove braces from approx_percentile method (Trino/Presto)
* 👌 Updating code due to code review changes.
2023-07-20 20:20:24 +05:30
vanshika18
c1c61c16df
fix #2936 : Support Usage connector for Trino ( #12483 )
2023-07-20 19:51:17 +05:30
Ayush Shah
ab1ec50c2c
Fixes Mssql Ntext, text and Image ( #12490 )
2023-07-20 13:34:35 +05:30
Mayur Singal
fee567d714
Fix #12301 : Fix vertica col comments for projections with the same name ( #12504 )
2023-07-20 13:18:18 +05:30
Onkar Ravgan
3e7e71d10a
Fixed bkp after fqnhash ( #12510 )
2023-07-20 12:00:06 +05:30
Mayur Singal
0c6435ae9b
Fix #2957 : Add support for qlik sense enterprise connector ( #12450 )
2023-07-19 21:08:05 +05:30
Onkar Ravgan
ceef44205b
added complex desc ( #12478 )
2023-07-19 12:01:12 +05:30
Mayur Singal
e79067d69a
Airflow add task type during ingestion ( #12310 )
2023-07-19 00:30:48 +05:30
vanshika18
b699acd900
Support Snowflake Database or Schema when has . in the name ( #12349 )
2023-07-18 21:12:12 +05:30
Pere Miquel Brull
44d9f62bba
Fix explore cache name ( #12441 )
2023-07-14 14:12:11 +02:00
Teddy
838ff73a4e
add sample data for table profile ( #12422 )
2023-07-14 09:13:22 +02:00
Teddy
42a426226e
Fixes Issue #11803 #12103 - Add BigQuery Struct Support ( #12435 )
...
* ref: implemented interface for profiler components + removed struct logic
* ref: ran python linting
* ref: added UML diagram to readme.md
* ref: empty commit for labeler check
* ref: remove multiple context manager for 3.7 3.8 compatibility
* ref: remove
* fix: mapper logic for BQ struct types
* feat: added BQ support for structs
* feat: clean code smell + handle null self.col.table value
* feat: ran python linting
* feat: updated test for profiler handler + disabled flaky test
* Update ingestion/tests/unit/profiler/pandas/test_sample.py
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-07-14 09:12:46 +02:00
vanshika18
bbe58b1b5f
Handle Table type for Snowflake Transient Tables ( #12274 )
...
* Handle Table type for Snowflake Transient Tables
* Clean the code by removing comments
* Optimize the code
* Optimize the code
* Clean the code
* Rename includeTempTables with includeTransientTables
* Rename includeTempTables to includeTransientTables
* Update Documentation and resolved migartion query error
* Resolved migartion query for postgres connection
---------
Co-authored-by: Vanshika Kabra <vanshikakabra@Vanshikas-MacBook-Pro.local>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-07-13 19:25:46 +02:00
Teddy
54fbe250a1
fix: import error + BQ E2E CLI ( #12420 )
2023-07-13 13:35:37 +02:00
Onkar Ravgan
e200a42926
Removed datamodels references from dashboard and datamodels ( #12371 )
2023-07-13 14:26:28 +05:30
Onkar Ravgan
90160ac761
Migrated DynamoDB to NoSQL type ( #12404 )
2023-07-13 14:25:31 +05:30
vanshika18
9174cea1ab
Add support for Superset datamodels ( #12183 )
2023-07-13 14:10:37 +05:30
Pere Miquel Brull
a3bff293a4
Fix #12190 - Bring support for Airflow 2.6 & Use Airflow 2.6.3 as the new ingestion base ( #12398 )
...
* Bump Airflow version
* Support Airflow 2.6
* Lint
* Bump airflow version
* Fix airflow 2.6 lineage
* Fix airflow 2.6 lineage
2023-07-13 06:41:59 +02:00
Teddy
b89cf64f14
Clean up profiler ( #12369 )
...
* ref: implemented interface for profiler components + removed struct logic
* ref: ran python linting
* ref: added UML diagram to readme.md
* ref: empty commit for labeler check
* ref: remove multiple context manager for 3.7 3.8 compatibility
* ref: remove
2023-07-12 17:02:32 +02:00
Pere Miquel Brull
1b7c5e3233
Fix #12321 - Deprecate impala scheme from Hive ( #12327 )
...
* Deprecate impala scheme from Hive
* Add migrations
2023-07-11 17:38:32 +02:00
Pere Miquel Brull
6a1b1190a7
Fix #12313 - Airflow lineage parsing and schedules ( #12339 )
...
* Fix docs
* Fix lineage and intervals
* Format
2023-07-11 07:35:22 +02:00
gauthk6
59c384f4e9
Fixes 11973: Pass required parameters while patching the description of a TestCase ( #12244 )
...
* Fixes a bug while patching the description of a TestCase
* Update docker-compose.yml
* Update docker-compose.yml
* Ran pre-commit checks and linter
* Added some clarifying points and fixed some grammatical errors in the documentation for installation instructions.
* revert changes made to docs
2023-07-07 09:51:23 +02:00
Ayush Shah
9ae3407be0
Fix Filter Pattern Delete issue ( #12293 )
2023-07-06 10:18:44 +05:30
Megumi AIKAWA
9aede5e821
Fixes 11924 : Add pgspider support ( #12159 )
...
* Add PGSpider service connector
* add sqlalchemy-pgspider to setup.py
* Delete unnecessary code and move files.
* Change to work as a Postgres connector sub-module
* Rebuild PGSpiderLineage class into a set of funcs
---------
Co-authored-by: Pham Ngoc Son <son1.phamngoc@toshiba.co.jp>
2023-07-05 09:18:59 +02:00
Hung Duong
64f147c517
Fixes #12129 : enhance bigquery sample data query ( #12130 )
...
* upgrade BigQuery Sampler
* beautify code
* revert old way of profiler & data quality, keep fetch new way sample
* Update profiler_source.py
* Update profiler_source.py
---------
Co-authored-by: hung.duong <hung.duong@be.com.vn>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2023-07-04 08:37:15 +02:00
Mohit Yadav
466e325e0a
Added Server migration table to utility ( #12263 )
...
* Added Server migration table to utility
* updated to be in not to migrate
* py checkstyle failure
2023-07-03 19:45:50 +02:00
Onkar Ravgan
48b98d0be8
default gen time for comp time ( #12256 )
2023-07-03 12:30:02 +05:30
Pere Miquel Brull
b59ff24531
Fix looker view parser ( #12254 )
2023-07-03 07:25:08 +02:00
Teddy
93d132de5c
fix: removed SampleDataConnection class from package ( #12229 )
2023-07-01 11:21:21 -07:00
Onkar Ravgan
2a5a355c5f
fixed dbt table level test ( #12247 )
2023-07-01 19:00:41 +02:00
Pere Miquel Brull
1ecf5607c7
Looker - Fix file extension and blob import ( #12232 )
...
* Fix file extension and blob import
* Fix file extension and blob import
2023-06-29 16:14:17 +02:00
Onkar Ravgan
c1a379e202
made tableau name fields optional ( #12223 )
2023-06-29 14:04:07 +05:30
Onkar Ravgan
acf25f4555
Added pagination Tableau data sources graphql ( #12187 )
...
* Added pagination tableau graphql
* changed downstream workbook
2023-06-28 14:57:09 +02:00
Teddy
caa78a2310
fix: updated MySQL and Postres conn in backup/restore ( #12191 )
...
* fix: updated MySQL and Postres conn in backup/restore
* fix: ran linting
2023-06-28 13:46:29 +05:30
Ayush Shah
9e0a84e904
Fix Profiler Workflow infinite loop ( #12185 )
...
* Fix Profiler Workflow infinite loop
2023-06-27 20:19:02 +05:30
Ayush Shah
81a152140c
fix connection kwargs error in profiler ( #12181 )
2023-06-27 19:44:53 +05:30
Pere Miquel Brull
c22c5871f8
Update logging messages for JWT tokens ( #12169 )
2023-06-27 12:32:08 +02:00
Teddy
45a5ff49c8
fix: check on undefined bool value of clause ( #12162 )
2023-06-26 20:31:37 +02:00
Pere Miquel Brull
a9135e8db5
Mask queries, Table sample data and Topic sample data ( #12139 )
...
* Filter out PII queries
* Mask topic PII
* Mask topic PII
* Update sample data test
* Format
* Moved logic to repository or PIIMasker
* Fix test
* chore(ui): update sample data api endpoint for topic
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-26 11:47:39 +02:00
Ayush Shah
29d3de2520
Move Top Level Imports ( #12145 )
2023-06-26 11:54:25 +05:30
Mayur Singal
a3fd6e9522
Fix #11659 : Add support for filter patterns in dbt workflow ( #12063 )
2023-06-26 11:30:35 +05:30
Mayur Singal
18c8eb318e
Databricks Support Table Constraints ( #12138 )
...
* Databricks Support Table Constraints
* pylint fix
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-06-26 07:01:53 +02:00
Mayur Singal
7d1b123efe
Fix atlas lineage custom db name issue ( #12108 )
...
* Fix atlas lineage custom db name issue
* Fix test
2023-06-25 19:16:00 -07:00
Onkar Ravgan
3a9d8a9a45
powerbi loop bug fix ( #12120 )
2023-06-25 18:20:59 +02:00
Pere Miquel Brull
97e08ee25c
Fix #12106 - Fix looker chart sourceUrl & Array datatype handling ( #12113 )
...
* Fix looker chart url
* Handle array datatype
* Handle array datatype
2023-06-25 08:18:36 -07:00
Aniket Katkar
67ffa1cce3
fix(UI): reverted the ES config step for 'Data Insights Pipeline' ( #12121 )
...
* Reverted the ES config step for Data Insights pipeline
* es sink fixes for ingestion
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-06-24 18:15:51 +05:30
