Harshal Sheth
d569734193
refactor(ingest): Tableau cleanup ( #6131 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-10 15:52:34 -07:00
Harshal Sheth
879894d702
feat(ingest): airflow - support enabled flag for airflow config ( #6089 )
2022-10-11 00:42:13 +02:00
Mayuri Nehate
7b88de89d5
fix(ingest): snowflake - allow profiling to work with geography type ( #6162 )
2022-10-10 08:05:09 -07:00
Harry Nguyen
e9f615442b
fix(ingest): bigquery - Fix syntax error in get_all_schema_tables_query ( #6159 )
...
Co-authored-by: Nguyen Tri Hieu <nhieu@fossil.com>
2022-10-10 10:39:30 +02:00
Shirshanka Das
e9c4c823d8
fix(ingest): bigquery-beta - ensure that status aspect is emitted for… ( #6154 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-08 16:00:45 -07:00
Harshal Sheth
30014072ad
feat(ingest): lookml - add support for local/remote dependencies ( #6150 )
2022-10-07 13:14:21 -07:00
Tamas Nemeth
0f04cd6431
fix(ingest): bigquery-beta - eliminate the need for data.read permission for table schema ( #6146 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-07 10:19:16 -07:00
Mayuri Nehate
b8513ae492
fix(ingest): fetch workbook tags in workbooks graphql query ( #6102 )
2022-10-07 15:04:03 +02:00
Pablo Takara
485d56982a
fix(ingestion): add fallback to trino ( #6044 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-10-06 14:14:45 -07:00
Harshal Sheth
f446ccef2e
fix(ingest): snowflake - use approx count distinct for columns, fix quoting ( #6107 )
2022-10-06 13:56:29 -07:00
Shirshanka Das
3106e42e89
fix(ingest): file - add configurability for counting all elements before starting ingestion ( #6136 )
2022-10-06 10:24:23 -07:00
Tamas Nemeth
6db0925b26
fix(ingest): bigquery-beta - fix for missing key error if dataset was empty ( #6133 )
2022-10-06 09:20:49 -07:00
Harshal Sheth
164e47a2e9
refactor(ingest): remove typing workarounds ( #6108 )
...
Possible now that we're on mypy 0.980.
2022-10-04 21:12:05 -07:00
Mayuri Nehate
4ee3ef14ef
fix(ingest): ignore usage and operation for snowflake datasets without schema ( #6112 )
2022-10-04 21:01:11 -07:00
Tamas Nemeth
2f79b50c24
fix(ingest): presto-on-hive - not failing on Hive type parsing error ( #6118 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-04 20:54:38 -07:00
Harshal Sheth
95afc1d73f
refactor(ingest): use pydantic utilities for NamingPattern ( #6013 )
...
* refactor(ingest): use pydantic utilities for NamingPattern
* clean up replacement logic
* flatten config hierarchy
2022-10-04 20:53:18 -07:00
Ravindra Lanka
055e4082da
fix(ingestion): fix percent change computation in stale_entity_removal ( #6121 )
2022-10-04 20:40:59 -07:00
Harshal Sheth
feb9d59b4d
fix(ingest): remove unused mysql golden file ( #6106 )
2022-10-04 20:26:51 -07:00
Harshal Sheth
3c0f63c50a
fix(ingest): hide deprecated path_spec option from config ( #5944 )
2022-10-04 12:14:00 -07:00
Thomas Tauber-Marshall
6b83cab82c
feat(ingest): add CorpUser and CorpGroup to the Python SDK ( #5930 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-04 12:13:38 -07:00
Roko Gudić
d52fde126f
feat(ingest): salesforce - add sandbox support ( #6105 )
...
Co-authored-by: rgudic <roko.gudic@infobip.com>
2022-10-04 09:05:53 -07:00
RyanHolstien
bfb903cfb8
feat(ingest): add async option to ingest proposal endpoint ( #6097 )
...
* feat(ingest): add async option to ingest proposal endpoint
* small tweak to validate before write to K, also keep existing path for timeseries aspects
* avoid double convert
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-03 19:56:19 -05:00
Tamas Nemeth
05f5c123bc
fix(ingest): bigquery-beta - Lowering a bit memory footprint of bigquery usage ( #6095 )
...
* Lowering a bit memory footprint of bigquery usage
* Filtering out not seen tables from usage generation
2022-10-01 14:39:04 +02:00
Shirshanka Das
3d8d8c6f8e
fix(ingest): lookml - extract column correctly ( #6093 )
2022-09-30 10:03:22 +02:00
Tamas Nemeth
a6930562fd
fix(ingest): presto-on-hive - Setting display name properly ( #6065 )
2022-09-29 16:20:17 +02:00
Harshal Sheth
71f5ec7f19
fix(ingest): mssql - alias for mssql-odbc source ( #6080 )
2022-09-29 15:33:49 +02:00
Harshal Sheth
05b9d4ef2f
fix(doc): note that Athena doesn't support lineage ( #6081 )
2022-09-29 15:30:09 +02:00
Shirshanka Das
74d9fa25a6
feat(ingest): looker - add support for simple column level lineage ( #6084 )
2022-09-28 20:41:30 -07:00
Mayuri Nehate
21a8718b10
feat(ingest): add column-level lineage support for snowflake ( #6034 )
2022-09-28 16:30:05 -07:00
Harshal Sheth
7e08a05a22
docs(ingest): clarify bigquery-beta multiproject setup ( #6071 )
2022-09-27 23:52:55 -07:00
Tamas Nemeth
3b9e9793a7
fix(ingest): bigquery-beta - handling complex types properly ( #6062 )
2022-09-27 21:31:24 +02:00
Harshal Sheth
7da2e9852f
fix(ingest): compat with mypy 0.981 ( #6056 )
2022-09-27 17:34:57 +02:00
Harshal Sheth
3f1d47c069
feat(ingest): list referenced env variables in recipe ( #6043 )
2022-09-26 23:16:18 -07:00
Tamas Nemeth
32b8bef92e
fix(ingest): bigquery-beta - Additional fixes for Bigquery beta ( #6051 )
2022-09-26 14:08:09 -07:00
Alexey Kravtsov
3c3ab64954
feat(ingest): implement compression for CheckpointState ( #6007 )
2022-09-26 10:18:42 -07:00
Harshal Sheth
611c053117
refactor(ingest): use aspect map in transformers ( #6040 )
2022-09-25 23:48:58 -07:00
Harshal Sheth
f227bd982b
refactor(ingest): streamline pydantic configs ( #6011 )
2022-09-25 23:37:48 -07:00
Tamas Nemeth
3b1a0c568e
fix(ingest): bigquery-beta - Getting datasets with biquery client ( #6039 )
...
* Getting datasets with biquery client instead of information schema because it did not work everywhere
Changing lists to lossylist in report
2022-09-24 00:48:15 +02:00
John Joyce
78454888b5
refactor(ingest): Browse Paths Upgrade Tableau ( #6008 )
2022-09-23 13:02:34 -07:00
Tamas Nemeth
722be5dc0c
fix(ingest): bigquery-beta - Project name quote and graceful lineage/usage failures ( #6035 )
2022-09-23 19:25:34 +02:00
RyanHolstien
0728656fbf
feat(patch): initial support of json patch style semantics in MCPs ( #5901 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-22 23:52:18 -07:00
Harshal Sheth
3a6d1d2bf1
feat(ingest): aws - support extra args to role config ( #6031 )
2022-09-22 17:43:32 -07:00
Harshal Sheth
27f28019de
refactor(ingest): move common host_port validation ( #6009 )
2022-09-22 16:32:07 -07:00
Ravindra Lanka
b8941ab190
feat(ingestion): Add fail-safe stale entity removal via configurable 'fail_safe_threshold' param. ( #6027 )
2022-09-22 16:09:22 -07:00
Harshal Sheth
2c03005fc2
fix(ingest): glue - better error handling ( #6030 )
2022-09-22 14:53:19 -07:00
Shirshanka Das
11191734fe
fix(ingest): lookml - make explore parsing more robust ( #6028 )
2022-09-22 13:14:39 -07:00
Harshal Sheth
25b335d3b3
feat(quickstart): add --no-pull-images option to docker quickstart ( #6017 )
2022-09-21 21:57:17 -07:00
Harshal Sheth
68db859ca1
refactor(ingest): streamline two-tier db config validation ( #5986 )
2022-09-21 10:45:37 -07:00
Ravindra Lanka
b638bcf442
feat(ingestion): Documentation on adding stateful ingestion use-cases to new sources ( #5985 )
2022-09-21 09:02:50 -07:00
mohdsiddique
44a67b52cf
feat(ingestion): Tableau dashboard & chart usage stats ( #5999 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 17:41:56 -07:00
