3975 Commits

Author SHA1 Message Date
Aseem Bansal
32e598df2b
feat(ingest): working with multiple bigquery projects (#5240) 2022-06-27 14:21:54 +05:30
Shirshanka Das
4b3135a0f8
feat(ingest): dbt - improving dbt_meta mapping (#5237) 2022-06-24 13:43:12 +02:00
Aseem Bansal
f66a6b41ef
fix(ingest): do not dump password (#5235) 2022-06-24 17:05:39 +05:30
Tamas Nemeth
25a0a17f51
fix(ingest): Handling gracefully sql parser error in bigquery lineage (#5238) 2022-06-24 15:11:01 +05:30
Jordan Wolinsky
2be1e64cde
fix(ingest): s3 - add check for 0 rows in profiling (#5219) 2022-06-23 12:01:59 +02:00
Tamas Nemeth
1086614129
fix(ingest): bigquery - Grouping date named tables at bigquery (#5230)
* Grouping date named tables at bigquery
* Fixing table name for sharded tables
2022-06-23 11:53:25 +02:00
Aseem Bansal
1adcc994f3
feat(ingest): remove need for sink block in UI based ingestion (#5208) 2022-06-23 15:22:40 +05:30
Piotr Sierkin
9f2b3b9ba6
fix(ingest): dbt - make assertion display names consistent (#5214)
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-06-22 22:07:00 -07:00
mayurinehate
022ef2f17c
feat(tableau): use pagination for all connection queries (#5204) 2022-06-22 08:53:31 -07:00
Shirshanka Das
141f33508c
fix(ingest): databricks - ingest structs correctly through hive (#5223) 2022-06-22 08:10:01 -07:00
Aseem Bansal
8c8f1b987a
fix(build): pin confluent kafka dependency (#5224) 2022-06-22 13:53:30 +05:30
Aditya Radhakrishnan
82ca92f8f9
feat(ingest): adds csv enricher ingestion source (#5221) 2022-06-22 12:25:39 +05:30
Atul Saurav
648ebfcc93
feat(ingestion) ldap: make ldap attrs keys configurable (#4682) 2022-06-21 12:01:00 -07:00
Ammar Farid
0949613df1
fix(ingest): add missing ownership types (#5209) 2022-06-21 08:04:50 -07:00
Pedro Silva
91d282e64c
feat(cli): Add token parameter for sample ingestion (#5160) 2022-06-21 07:15:57 -07:00
Tamas Nemeth
393c07ee52
refactor(ingest): bigquery-usage - Adding tests for bigquery usage filters (#5195) 2022-06-20 18:28:27 -07:00
Aseem Bansal
c942350e3c
feat(ci): split integration tests to reduce run time (#5205) 2022-06-20 21:08:37 +05:30
Aseem Bansal
d518b5a085
fix(cli): correct handling of env variables (#5203) 2022-06-20 20:53:47 +05:30
Shirshanka Das
0ee2569d5c
feat(ingest): dbt - adding support for dbt tests (#5201) 2022-06-18 13:30:04 -07:00
BALyons
e360d267be
fix(ingest): correct case of sys views for mssql description population 2022-06-17 21:46:02 +05:30
Dmitry Bryazgin
9d7d7deac2
fix(ingest): If there is no manager for a LDAP user don't fail (#5180) 2022-06-17 21:45:36 +05:30
Aseem Bansal
4541379024
feat(build): changes to decrease build time, cancel runs in case of multiple commits (#5187) 2022-06-17 18:05:10 +05:30
Tamas Nemeth
b0db17e237
feat(ingest): bigquery - option to send usage queries as well as operational metadata (#5151) 2022-06-16 16:00:35 -07:00
data ** 100
f4674c3e78
fix(ingest): glue - fix profiling config option for column_count (#5178) 2022-06-15 23:07:58 -07:00
Sayak Maity
0257388e65
fix(airflow): fixes DeprecationWarning with hook-class-names (#5143) 2022-06-13 13:32:11 +02:00
mayurinehate
7b143b06fc
feat(ingest): snowflake profile tables only if they have been updates since N days (#5132) 2022-06-13 14:59:16 +05:30
skrydal
503208beaa
feat(ingest): dbt - add platform instance for target platform (#5129) 2022-06-12 09:29:44 -07:00
mayurinehate
7aa481dd07
fix(ingest): fix table urn for athena connectionType (#5135) 2022-06-09 11:10:23 -07:00
BZ
01a96a029d
fix(dbt): validator should not complain when paths are None (#5125) 2022-06-09 08:44:46 -07:00
Tamas Nemeth
6302d32685
fix(ingestion): bigquery-usage - Only send operational metadata for allowed tables (#5127) 2022-06-09 08:38:23 +02:00
cccs-eric
f25a5f043d
fix(iceberg): Change how MapType are mapped to Avro to support complex Map key type. (#5060) 2022-06-09 08:13:10 +05:30
Sebo Kim
da273802be
fix(ingest): looker - original view name should be used for explore_joins (#4928) 2022-06-08 18:42:55 -07:00
Tamas Nemeth
09fc50674d
fix(ingestion): bigquery-usage - Removing filtering at queryevents (#5124) 2022-06-08 21:21:58 +02:00
Aseem Bansal
2b06cb19fd
fix(bigquery): handling of empty partitioned tables, improve report message (#5122) 2022-06-08 23:03:32 +05:30
Tamas Nemeth
c677a06fd8
feat(ingestion): bigquery-usage - Collect stats from read event reasons (#5118) 2022-06-08 14:18:22 +02:00
Gabe Lyons
538cfba526
fix(ingest): dbt - ensure platform_instance filled out only for dbt nodes (#5115) 2022-06-07 20:09:42 -07:00
Shirshanka Das
601ee9eae1
feat(looker): ensure explore name matches looker's display name (#5111) 2022-06-07 16:49:53 -07:00
Tamas Nemeth
039e98e8cf
feat(docs): Updating example files with the new ingestion recipe suffix (#5103) 2022-06-07 15:52:26 -07:00
John Joyce
5c501fba2c
fix(cli): Bumping avro-gen3 to 0.7.4 (#5098) 2022-06-07 12:56:06 -07:00
Shirshanka Das
51854a768d
fix(ci): pin google-cloud-logging to avoid pip backtracking (#5109) 2022-06-07 23:14:00 +05:30
Aseem Bansal
596853ae1f
fix(build): remove dependencies added for compatibility (#5108) 2022-06-07 19:59:17 +05:30
Aseem Bansal
e567d70d37
fix(build): reduce time taken for resolution (#5106) 2022-06-07 17:32:33 +05:30
Shirshanka Das
7cef07490f
fix(deps): ingest - moving packaging to framework_common (#5096) 2022-06-06 16:34:23 -07:00
Aseem Bansal
c43ad54a64
fix(profiling): don't stop if some steps fail (#5095) 2022-06-06 14:36:32 -07:00
Gabe Lyons
3c7c83f447
tests(lookml): adding tests for model deny patterns (#4934) 2022-06-06 08:26:27 -07:00
Lal Rishav
4f82e29697
feat(ingest): snowflake using oauth for Azure (#4647) 2022-06-06 17:49:49 +05:30
Shirshanka Das
e62c647693
feat(cli): suggest upgrades when appropriate (#5091) 2022-06-04 09:34:17 -07:00
Tamas Nemeth
72d6c57044
feat(ingest): rest_emitter - Adding option to rest emitter to disable ssl verification (#5042) 2022-06-04 12:43:50 +05:30
Shirshanka Das
64fae6121a
feat(docs): ingest - add tab for config json schema (#5086) 2022-06-03 14:57:36 -07:00
Tamas Nemeth
aeffd26bb3
fix(ingest): looker - handle invalid json in Looker query dynamic field (#5083) 2022-06-03 14:30:57 -07:00