John Joyce
|
2e3a25123d
|
refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker (#6002)
|
2022-12-21 08:02:59 -08:00 |
|
Harshal Sheth
|
137f4500b6
|
feat(ingest/stateful): remove platform_instance_id from state urn (#6795)
|
2022-12-20 12:12:19 -05:00 |
|
Harshal Sheth
|
5584bfb469
|
refactor(ingest/stateful): remove get_last_state method (#6794)
|
2022-12-19 20:48:22 -05:00 |
|
Harshal Sheth
|
e9d50ed992
|
refactor(ingest/stateful): remove IngestionJobStateProvider (#6792)
|
2022-12-19 17:03:54 -05:00 |
|
Harshal Sheth
|
47be95689e
|
refactor(ingest/stateful): remove most remaining state classes (#6791)
|
2022-12-19 13:40:48 -05:00 |
|
Tamas Nemeth
|
e41b455e14
|
fix(ingest): bigquery - sharded table support improvements (#6789)
|
2022-12-19 18:57:37 +01:00 |
|
Mayuri Nehate
|
9716a49067
|
fix(ingest): correct external url for account identifier with account name (#6715)
|
2022-12-16 14:00:42 -05:00 |
|
Harshal Sheth
|
8a537b0559
|
feat(ingest): add datahub state inspect command (#6763)
|
2022-12-15 18:55:36 -05:00 |
|
Harshal Sheth
|
6152b5e9f7
|
feat(ingest): simplify more stateful ingestion state (#6762)
|
2022-12-15 11:33:29 -05:00 |
|
Shirshanka Das
|
db182e4639
|
fix(python-sdk): DataHubGraph get_aspect should accept empty responses (#6760)
|
2022-12-14 10:40:16 -08:00 |
|
Harshal Sheth
|
2f95719dba
|
feat(ingest): remove source config from DatahubIngestionCheckpoint (#6722)
|
2022-12-14 12:39:21 -05:00 |
|
Patrick Franco Braz
|
f0a371941e
|
refactor(ingest): bigquery-lineage - allow tables and datasets in uppercase (#6739)
|
2022-12-14 14:58:03 +01:00 |
|
Harshal Sheth
|
cf3db168ac
|
feat(ingest): start simplifying stateful ingestion state (#6740)
|
2022-12-13 10:05:57 +01:00 |
|
Harshal Sheth
|
7d63399d00
|
fix(ingest): fix serde for empty dicts in unions with null (#6745)
The code changes in https://github.com/acryldata/avro_gen/pull/16, but tests are written here.
|
2022-12-13 08:17:24 +01:00 |
|
Dmitry Bryazgin
|
551ef1b335
|
feat(ingest): add stateful ingestion to the ldap source (#6127)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-12-13 01:13:39 -05:00 |
|
Mayuri Nehate
|
65ba13d9aa
|
feat(ingest): snowflake - add separate config for include_column_lineage in snowflake (#6712)
|
2022-12-12 15:23:12 +01:00 |
|
Harshal Sheth
|
fd911c9820
|
feat(ingest): redact configs reported in ingestion_run_summary (#6696)
|
2022-12-12 10:48:26 +01:00 |
|
Harshal Sheth
|
b7735d5b21
|
fix(ingest): fix bug in auto_status_aspect (#6705)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
|
2022-12-09 12:24:39 -05:00 |
|
Harshal Sheth
|
4dded454ff
|
fix(ingest): cleanup config extra usage (#6699)
|
2022-12-08 16:34:34 -08:00 |
|
Mayuri Nehate
|
eeb7a9dfe5
|
feat(ingest): snowflake - update snowflake docs, add simple validations (#6636)
|
2022-12-07 14:56:03 +01:00 |
|
Tamas Nemeth
|
9a1f78fc60
|
fix(ingest): profiling - Changing profiling defaults on Bigquery and Snowflake (#6640)
|
2022-12-07 10:33:10 +01:00 |
|
Harshal Sheth
|
a1e62c723e
|
docs(ingest): add airflow docs that use the PythonVirtualenvOperator (#6604)
|
2022-12-02 19:56:17 +01:00 |
|
Harshal Sheth
|
44cfd21a65
|
chore(ingest): bump and pin mypy (#6584)
|
2022-12-02 19:53:28 +01:00 |
|
Tamas Nemeth
|
8d525d67a9
|
fix(ingest): kafka - properly picking doc from union type (#6472)
|
2022-11-23 20:42:21 +01:00 |
|
Mayuri Nehate
|
22847a987a
|
feat(ingest): automated term classification for snowflake (#6376)
|
2022-11-23 00:43:30 -05:00 |
|
Harshal Sheth
|
74cc88f2df
|
fix(ingest): correctly handle transformer patch semantics (#6505)
|
2022-11-22 09:29:57 -08:00 |
|
Harshal Sheth
|
05a0f3e2a6
|
feat(ingest): dbt cloud integration (#6323)
|
2022-11-21 14:14:33 -05:00 |
|
Harshal Sheth
|
3e907ab0d1
|
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ (#6204)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
|
2022-11-11 15:04:36 -05:00 |
|
Harshal Sheth
|
1a81d8de6a
|
feat(ingest): supports MCEs in domain transformer (#6364)
|
2022-11-05 11:41:43 -07:00 |
|
Harshal Sheth
|
0ca3383d30
|
feat(ingest): support reserved keywords in model codegen (#6351)
|
2022-11-02 22:26:56 -07:00 |
|
Harshal Sheth
|
b4687ffceb
|
feat(ingest): drop plugin support for airflow 1.x (#6331)
|
2022-11-01 21:12:34 -07:00 |
|
Harshal Sheth
|
ef824bd082
|
feat(ingest): add fallthrough support to KeyValuePattern (#6302)
|
2022-10-28 11:07:47 +02:00 |
|
Tamas Nemeth
|
9015a43f25
|
fix(ingest): bigquery-beta - Adding python 3.8 fix for memory footprint util (#6228)
|
2022-10-18 17:59:31 -07:00 |
|
Harshal Sheth
|
d08f5f7cdd
|
feat(ingest): replace base85's pickle with json (#6178)
|
2022-10-14 14:48:44 -07:00 |
|
Harshal Sheth
|
09616ee2b3
|
feat(ingest): include instance in container dataPlatform when provided (#6083)
|
2022-10-13 11:29:54 -07:00 |
|
Tamas Nemeth
|
6e34cd6001
|
feat(ingest): bigquery-beta - Parsing view ddl definition for lineage (#6187)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
|
2022-10-12 18:24:04 -07:00 |
|
Harshal Sheth
|
95d1e01195
|
feat(ingest): infer aspect name from type in get_aspect (#6033)
|
2022-10-11 13:35:41 -07:00 |
|
Mayuri Nehate
|
7b88de89d5
|
fix(ingest): snowflake - allow profiling to work with geography type (#6162)
|
2022-10-10 08:05:09 -07:00 |
|
Shirshanka Das
|
e9c4c823d8
|
fix(ingest): bigquery-beta - ensure that status aspect is emitted for… (#6154)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-10-08 16:00:45 -07:00 |
|
Tamas Nemeth
|
2f79b50c24
|
fix(ingest): presto-on-hive - not failing on Hive type parsing error (#6118)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
|
2022-10-04 20:54:38 -07:00 |
|
Ravindra Lanka
|
055e4082da
|
fix(ingestion): fix percent change computation in stale_entity_removal (#6121)
|
2022-10-04 20:40:59 -07:00 |
|
Tamas Nemeth
|
3b9e9793a7
|
fix(ingest): bigquery-beta - handling complex types properly (#6062)
|
2022-09-27 21:31:24 +02:00 |
|
Harshal Sheth
|
3f1d47c069
|
feat(ingest): list referenced env variables in recipe (#6043)
|
2022-09-26 23:16:18 -07:00 |
|
Alexey Kravtsov
|
3c3ab64954
|
feat(ingest): implement compression for CheckpointState (#6007)
|
2022-09-26 10:18:42 -07:00 |
|
Harshal Sheth
|
27f28019de
|
refactor(ingest): move common host_port validation (#6009)
|
2022-09-22 16:32:07 -07:00 |
|
Ravindra Lanka
|
b8941ab190
|
feat(ingestion): Add fail-safe stale entity removal via configurable 'fail_safe_threshold' param. (#6027)
|
2022-09-22 16:09:22 -07:00 |
|
Harshal Sheth
|
68db859ca1
|
refactor(ingest): streamline two-tier db config validation (#5986)
|
2022-09-21 10:45:37 -07:00 |
|
Mayuri Nehate
|
b195b6c123
|
fix(ingest): encode reserved characters when creating dataset urn (#5977)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-09-20 16:59:02 -07:00 |
|
Harshal Sheth
|
937ab192c0
|
feat(ingest): add support for aliases in plugin registry (#5958)
|
2022-09-16 07:19:32 -07:00 |
|
skrydal
|
a026c84691
|
feat: qualifiedName support + populating glue ARN (#5952)
|
2022-09-15 21:15:03 -07:00 |
|