3904 Commits

Author SHA1 Message Date
Aseem Bansal
60714df08f
fix(bigquery): multi-project GCP setup run query through correct project (#5393) 2022-07-14 16:03:01 +05:30
Tamas Nemeth
4334248953
feat(airflow): Circuit breaker and python api for Assertion and Operation (#5196) 2022-07-13 10:17:38 -07:00
Tamas Nemeth
9ec4fbae86
fix(ingest): bigquery - Graceful bq partition id date parsing failure (#5386) 2022-07-13 13:21:45 +02:00
Tamas Nemeth
4c6d427800
fix(cli): respect server telemetry settings correctly (#5384)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-13 11:24:18 +02:00
Tamas Nemeth
f0281f32ab
fix(airflow): fix for failing serialisation when Param was specified + support for external task sensor (#5368)
fixes #4546
2022-07-12 10:20:27 -07:00
Aezo
ff0aa3f24b
feat(sdk): python - add get_aspects_for_entity (#5255)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-12 10:10:07 -07:00
Navin Sharma
f3e5afdba9
feat(quickstart): Adding env variables and cli options for customizing mapped ports in quickstart (#5353)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-11 22:33:24 -07:00
Shirshanka Das
489b5bb5b4
fix(ingest): domains - check whether urn based domain exists during resolution (#5373) 2022-07-11 15:08:26 -07:00
Shirshanka Das
860d475c5e
feat(ingest): improve domain ingestion usability (#5366) 2022-07-11 09:37:38 -07:00
Shirshanka Das
558a65a3c3
fix(ci): fix mysql test and attempt kafka-connect ingestion (#5352) 2022-07-07 08:28:34 -07:00
Mugdha Hardikar
94dd3ad5a1
fix(ingest): bigquery-usage - dataset name for sharded tables (#5347)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-07 00:37:13 -07:00
Mayuri Nehate
4b515e022b
feat(ingest): salesforce - add connector (#5104)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
2022-07-06 10:01:16 -07:00
Aditya Radhakrishnan
fc8e59387d
feat(ingest): update CSV source to support description and ownership type (#5346) 2022-07-06 21:29:29 +05:30
Mugdha Hardikar
77eaab609d
docs(bigquery): profiling report enhancement (#5342) 2022-07-06 18:38:35 +05:30
Vincent Koc
d60f789519
chore: Refactor Python Codebase (#5113) 2022-07-06 16:09:27 +05:30
Mugdha Hardikar
5216d72f91
feat(bigquery): support size, rowcount, lastmodified based table selection for profiling (#5329)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-06 15:29:26 +05:30
Mayuri Nehate
f23c2c9b64
fix(ingest): tableau - omit schema fields when name is absent (#5275) 2022-07-05 10:40:55 -07:00
Shirshanka Das
e93e4691fb
feat(ingest): lookml - adding support for only emitting reachable views from explores (#5333) 2022-07-05 10:14:12 -07:00
Mayuri Nehate
80e91b457f
feat(ingest): looker - ingest dashboard create/update/delete timestamps (#5312) 2022-07-05 09:04:41 -07:00
Tamas Nemeth
f08c3f784f
fix(ingest): bigquery - Fix for bigquery error when there was no bigquery catalog specified (#5303) 2022-07-01 17:47:07 +02:00
Shirshanka Das
5f0c554ab5
fix(cli): timeline - category should be owner not ownership (#5304) 2022-06-30 16:01:59 -07:00
Aseem Bansal
591a7d8a82
fix(cli): suppress secrets in stacktraces (#5302) 2022-06-30 22:53:30 +05:30
Tamas Nemeth
c1f8227693
fix(ingestion): Reverting sqlalchemy upgrade because it caused issues with mssql and redshift-usage (#5289) 2022-06-29 18:23:36 +02:00
Tamas Nemeth
d812fd18fa
fix(ingest) redshift: Adding missing dependencies and relaxing sqlalchemy dependency (#5284)
Relaxing sqlalchemy deps to make our plugins work with Airflow 2.3
2022-06-29 16:25:48 +02:00
Tamas Nemeth
fb1b1db7f7
fix(ingestion): profiling - Fixing partitioned table profiling in BQ (#5283) 2022-06-29 14:00:52 +02:00
Mugdha Hardikar
b76005d640
fix(ingest): s3 - Remove unneeded methods from s3_util (#5276) 2022-06-28 16:50:41 +02:00
Mugdha Hardikar
d62c77f70a
fix(ingest): delta-lake - fix dependency issue for snowflake due to s3_util (#5274) 2022-06-28 12:18:09 +02:00
Mugdha Hardikar
5455122267
feat(ingest): delta-lake: adding support for delta lake (#5259)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-06-27 16:16:33 -07:00
Tamas Nemeth
60ff0f45ee
feat(ingest): Option to define path spec for Redshift lineage generation (#5256) 2022-06-27 21:21:13 +05:30
Aseem Bansal
32e598df2b
feat(ingest): working with multiple bigquery projects (#5240) 2022-06-27 14:21:54 +05:30
Shirshanka Das
4b3135a0f8
feat(ingest): dbt - improving dbt_meta mapping (#5237) 2022-06-24 13:43:12 +02:00
Aseem Bansal
f66a6b41ef
fix(ingest): do not dump password (#5235) 2022-06-24 17:05:39 +05:30
Tamas Nemeth
25a0a17f51
fix(ingest): Handling gracefully sql parser error in bigquery lineage (#5238) 2022-06-24 15:11:01 +05:30
Jordan Wolinsky
2be1e64cde
fix(ingest): s3 - add check for 0 rows in profiling (#5219) 2022-06-23 12:01:59 +02:00
Tamas Nemeth
1086614129
fix(ingest): bigquery - Grouping date named tables at bigquery (#5230)
* Grouping date named tables at bigquery
* Fixing table name for sharded tables
2022-06-23 11:53:25 +02:00
Aseem Bansal
1adcc994f3
feat(ingest): remove need for sink block in UI based ingestion (#5208) 2022-06-23 15:22:40 +05:30
Piotr Sierkin
9f2b3b9ba6
fix(ingest): dbt - make assertion display names consistent (#5214)
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-06-22 22:07:00 -07:00
mayurinehate
022ef2f17c
feat(tableau): use pagination for all connection queries (#5204) 2022-06-22 08:53:31 -07:00
Shirshanka Das
141f33508c
fix(ingest): databricks - ingest structs correctly through hive (#5223) 2022-06-22 08:10:01 -07:00
Aseem Bansal
8c8f1b987a
fix(build): pin confluent kafka dependency (#5224) 2022-06-22 13:53:30 +05:30
Aditya Radhakrishnan
82ca92f8f9
feat(ingest): adds csv enricher ingestion source (#5221) 2022-06-22 12:25:39 +05:30
Atul Saurav
648ebfcc93
feat(ingestion) ldap: make ldap attrs keys configurable (#4682) 2022-06-21 12:01:00 -07:00
Ammar Farid
0949613df1
fix(ingest): add missing ownership types (#5209) 2022-06-21 08:04:50 -07:00
Pedro Silva
91d282e64c
feat(cli): Add token parameter for sample ingestion (#5160) 2022-06-21 07:15:57 -07:00
Tamas Nemeth
393c07ee52
refactor(ingest): bigquery-usage - Adding tests for bigquery usage filters (#5195) 2022-06-20 18:28:27 -07:00
Aseem Bansal
c942350e3c
feat(ci): split integration tests to reduce run time (#5205) 2022-06-20 21:08:37 +05:30
Aseem Bansal
d518b5a085
fix(cli): correct handling of env variables (#5203) 2022-06-20 20:53:47 +05:30
Shirshanka Das
0ee2569d5c
feat(ingest): dbt - adding support for dbt tests (#5201) 2022-06-18 13:30:04 -07:00
BALyons
e360d267be
fix(ingest): correct case of sys views for mssql description population 2022-06-17 21:46:02 +05:30
Dmitry Bryazgin
9d7d7deac2
fix(ingest): If there is no manager for a LDAP user don't fail (#5180) 2022-06-17 21:45:36 +05:30