Tamas Nemeth
d08a8f79ab
feat(ingest): preflight - Add way to check/upgrade brew package version in preflight if needed ( #5435 )
2022-07-22 14:20:12 +05:30
Guillaume Gardey
6101759c0a
fix(ingest/trino): Avoid exception if $properties table empty or not readable ( #5447 )
...
Under some configuration of access rules in Trino, the user may not have
read access to the content of the table, which will result in an exception
(`fetchone()` returns `None`)
This commit ensures no exception are raised and the ingestion can proceed.
2022-07-22 14:17:29 +05:30
Shirshanka Das
7ed9cd2838
feat(ingest): snowflake - basic test connection capability ( #5464 )
2022-07-22 09:14:37 +02:00
Mayuri Nehate
fa46167dfe
fix(ingest): tableau - fix tableau db error, add more logs ( #5423 )
2022-07-21 08:54:16 -07:00
Shirshanka Das
14d764a26f
fix(ingest): fix serialization of report to handle nesting ( #5455 )
2022-07-20 18:25:07 -07:00
Mugdha Hardikar
1f4d3722e2
fix(ingest): hive - add column comment as a column description ( #5449 )
2022-07-20 12:32:02 -07:00
Aseem Bansal
acb9879eb4
feat(cli,build): remove deprecated variables GMS_HOST/_PORT ( #5451 )
2022-07-20 20:54:43 +05:30
Mugdha Hardikar
ced6c38239
fix(ingest): bigquery-usage - fix dataset name for sharded table ( #5412 )
2022-07-19 20:59:02 -07:00
Tim Costa
d065553958
fix(ingest/glue): handle error when generating s3 tags for virtual view tables ( #5398 )
...
Co-authored-by: Tim Costa <timcosta@amazon.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-19 20:43:09 -07:00
Shirshanka Das
76133c6f37
feat(ingest): add test source connection feature, structured report file ( #5442 )
2022-07-19 20:40:59 -07:00
Mugdha Hardikar
a6dc669891
docs(bigquery): add changelog and unittest for profiling limits ( #5407 )
2022-07-19 09:39:09 +05:30
Mayuri Nehate
65a308a4ed
fix(ingest): looker - extract usage for dashboards allowed by pattern ( #5424 )
2022-07-18 14:21:17 +02:00
Mayuri Nehate
e6ce4324c9
fix(ingest): looker - pass transport options to all api calls ( #5417 )
2022-07-17 13:55:20 -07:00
Shirshanka Das
f387fa6149
fix(ingest): ensure upgrade checks run async ( #5383 )
2022-07-16 17:38:33 -07:00
Pedro Silva
bc96cf22e7
fix(metadata-ingestion) Fix broken csv enricher test ( #5406 )
2022-07-15 19:00:36 +05:30
Harshal Sheth
fdea718e38
build: improve comprehensiveness of gradle clean ( #5003 )
2022-07-15 15:38:32 +05:30
Mayuri Nehate
2c48329810
feat(model): dashboard usage model, is_null condition added ( #5397 )
2022-07-15 15:37:06 +05:30
Pedro Silva
b2edd44b6a
Adds support for Domains in CSV source ( #5372 )
2022-07-15 14:20:41 +05:30
Aseem Bansal
ef02bf4b04
fix(quickstart): use platform agnostic way to get folder ( #5400 )
2022-07-15 14:15:24 +05:30
Felix Lüdin
a0303448ba
feat(dashboards): add datasets field to DashboardInfo aspect ( #5188 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-07-14 09:54:02 -07:00
Aseem Bansal
45315ef1d4
fix(bigquery): add storage project name ( #5395 )
2022-07-14 19:52:38 +05:30
Aseem Bansal
60714df08f
fix(bigquery): multi-project GCP setup run query through correct project ( #5393 )
2022-07-14 16:03:01 +05:30
Tamas Nemeth
4334248953
feat(airflow): Circuit breaker and python api for Assertion and Operation ( #5196 )
2022-07-13 10:17:38 -07:00
Tamas Nemeth
9ec4fbae86
fix(ingest): bigquery - Graceful bq partition id date parsing failure ( #5386 )
2022-07-13 13:21:45 +02:00
Tamas Nemeth
4c6d427800
fix(cli): respect server telemetry settings correctly ( #5384 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-13 11:24:18 +02:00
Tamas Nemeth
f0281f32ab
fix(airflow): fix for failing serialisation when Param was specified + support for external task sensor ( #5368 )
...
fixes #4546
2022-07-12 10:20:27 -07:00
Aezo
ff0aa3f24b
feat(sdk): python - add get_aspects_for_entity ( #5255 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-12 10:10:07 -07:00
Navin Sharma
f3e5afdba9
feat(quickstart): Adding env variables and cli options for customizing mapped ports in quickstart ( #5353 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-11 22:33:24 -07:00
Shirshanka Das
489b5bb5b4
fix(ingest): domains - check whether urn based domain exists during resolution ( #5373 )
2022-07-11 15:08:26 -07:00
Shirshanka Das
860d475c5e
feat(ingest): improve domain ingestion usability ( #5366 )
2022-07-11 09:37:38 -07:00
Shirshanka Das
558a65a3c3
fix(ci): fix mysql test and attempt kafka-connect ingestion ( #5352 )
2022-07-07 08:28:34 -07:00
Mugdha Hardikar
94dd3ad5a1
fix(ingest): bigquery-usage - dataset name for sharded tables ( #5347 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-07 00:37:13 -07:00
Mayuri Nehate
4b515e022b
feat(ingest): salesforce - add connector ( #5104 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
2022-07-06 10:01:16 -07:00
Aditya Radhakrishnan
fc8e59387d
feat(ingest): update CSV source to support description and ownership type ( #5346 )
2022-07-06 21:29:29 +05:30
Mugdha Hardikar
77eaab609d
docs(bigquery): profiling report enhancement ( #5342 )
2022-07-06 18:38:35 +05:30
Vincent Koc
d60f789519
chore: Refactor Python Codebase ( #5113 )
2022-07-06 16:09:27 +05:30
Mugdha Hardikar
5216d72f91
feat(bigquery): support size, rowcount, lastmodified based table selection for profiling ( #5329 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-06 15:29:26 +05:30
Mayuri Nehate
f23c2c9b64
fix(ingest): tableau - omit schema fields when name is absent ( #5275 )
2022-07-05 10:40:55 -07:00
Shirshanka Das
e93e4691fb
feat(ingest): lookml - adding support for only emitting reachable views from explores ( #5333 )
2022-07-05 10:14:12 -07:00
Mayuri Nehate
80e91b457f
feat(ingest): looker - ingest dashboard create/update/delete timestamps ( #5312 )
2022-07-05 09:04:41 -07:00
Tamas Nemeth
f08c3f784f
fix(ingest): bigquery - Fix for bigquery error when there was no bigquery catalog specified ( #5303 )
2022-07-01 17:47:07 +02:00
Shirshanka Das
5f0c554ab5
fix(cli): timeline - category should be owner not ownership ( #5304 )
2022-06-30 16:01:59 -07:00
Aseem Bansal
591a7d8a82
fix(cli): suppress secrets in stacktraces ( #5302 )
2022-06-30 22:53:30 +05:30
Tamas Nemeth
c1f8227693
fix(ingestion): Reverting sqlalchemy upgrade because it caused issues with mssql and redshift-usage ( #5289 )
2022-06-29 18:23:36 +02:00
Tamas Nemeth
d812fd18fa
fix(ingest) redshift: Adding missing dependencies and relaxing sqlalchemy dependency ( #5284 )
...
Relaxing sqlalchemy deps to make our plugins work with Airflow 2.3
2022-06-29 16:25:48 +02:00
Tamas Nemeth
fb1b1db7f7
fix(ingestion): profiling - Fixing partitioned table profiling in BQ ( #5283 )
2022-06-29 14:00:52 +02:00
Mugdha Hardikar
b76005d640
fix(ingest): s3 - Remove unneeded methods from s3_util ( #5276 )
2022-06-28 16:50:41 +02:00
Mugdha Hardikar
d62c77f70a
fix(ingest): delta-lake - fix dependency issue for snowflake due to s3_util ( #5274 )
2022-06-28 12:18:09 +02:00
Mugdha Hardikar
5455122267
feat(ingest): delta-lake: adding support for delta lake ( #5259 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-06-27 16:16:33 -07:00
Tamas Nemeth
60ff0f45ee
feat(ingest): Option to define path spec for Redshift lineage generation ( #5256 )
2022-06-27 21:21:13 +05:30
