Mayuri Nehate
|
65ba13d9aa
|
feat(ingest): snowflake - add separate config for include_column_lineage in snowflake (#6712)
|
2022-12-12 15:23:12 +01:00 |
|
Harshal Sheth
|
fd911c9820
|
feat(ingest): redact configs reported in ingestion_run_summary (#6696)
|
2022-12-12 10:48:26 +01:00 |
|
Harshal Sheth
|
b7735d5b21
|
fix(ingest): fix bug in auto_status_aspect (#6705)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
|
2022-12-09 12:24:39 -05:00 |
|
Harshal Sheth
|
4dded454ff
|
fix(ingest): cleanup config extra usage (#6699)
|
2022-12-08 16:34:34 -08:00 |
|
Mayuri Nehate
|
eeb7a9dfe5
|
feat(ingest): snowflake - update snowflake docs, add simple validations (#6636)
|
2022-12-07 14:56:03 +01:00 |
|
Tamas Nemeth
|
9a1f78fc60
|
fix(ingest): profiling - Changing profiling defaults on Bigquery and Snowflake (#6640)
|
2022-12-07 10:33:10 +01:00 |
|
Harshal Sheth
|
a1e62c723e
|
docs(ingest): add airflow docs that use the PythonVirtualenvOperator (#6604)
|
2022-12-02 19:56:17 +01:00 |
|
Harshal Sheth
|
44cfd21a65
|
chore(ingest): bump and pin mypy (#6584)
|
2022-12-02 19:53:28 +01:00 |
|
Tamas Nemeth
|
8d525d67a9
|
fix(ingest): kafka - properly picking doc from union type (#6472)
|
2022-11-23 20:42:21 +01:00 |
|
Mayuri Nehate
|
22847a987a
|
feat(ingest): automated term classification for snowflake (#6376)
|
2022-11-23 00:43:30 -05:00 |
|
Harshal Sheth
|
74cc88f2df
|
fix(ingest): correctly handle transformer patch semantics (#6505)
|
2022-11-22 09:29:57 -08:00 |
|
Harshal Sheth
|
05a0f3e2a6
|
feat(ingest): dbt cloud integration (#6323)
|
2022-11-21 14:14:33 -05:00 |
|
Harshal Sheth
|
3e907ab0d1
|
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ (#6204)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
|
2022-11-11 15:04:36 -05:00 |
|
Harshal Sheth
|
1a81d8de6a
|
feat(ingest): supports MCEs in domain transformer (#6364)
|
2022-11-05 11:41:43 -07:00 |
|
Harshal Sheth
|
0ca3383d30
|
feat(ingest): support reserved keywords in model codegen (#6351)
|
2022-11-02 22:26:56 -07:00 |
|
Harshal Sheth
|
b4687ffceb
|
feat(ingest): drop plugin support for airflow 1.x (#6331)
|
2022-11-01 21:12:34 -07:00 |
|
Harshal Sheth
|
ef824bd082
|
feat(ingest): add fallthrough support to KeyValuePattern (#6302)
|
2022-10-28 11:07:47 +02:00 |
|
Tamas Nemeth
|
9015a43f25
|
fix(ingest): bigquery-beta - Adding python 3.8 fix for memory footprint util (#6228)
|
2022-10-18 17:59:31 -07:00 |
|
Harshal Sheth
|
d08f5f7cdd
|
feat(ingest): replace base85's pickle with json (#6178)
|
2022-10-14 14:48:44 -07:00 |
|
Harshal Sheth
|
09616ee2b3
|
feat(ingest): include instance in container dataPlatform when provided (#6083)
|
2022-10-13 11:29:54 -07:00 |
|
Tamas Nemeth
|
6e34cd6001
|
feat(ingest): bigquery-beta - Parsing view ddl definition for lineage (#6187)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
|
2022-10-12 18:24:04 -07:00 |
|
Harshal Sheth
|
95d1e01195
|
feat(ingest): infer aspect name from type in get_aspect (#6033)
|
2022-10-11 13:35:41 -07:00 |
|
Mayuri Nehate
|
7b88de89d5
|
fix(ingest): snowflake - allow profiling to work with geography type (#6162)
|
2022-10-10 08:05:09 -07:00 |
|
Shirshanka Das
|
e9c4c823d8
|
fix(ingest): bigquery-beta - ensure that status aspect is emitted for… (#6154)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-10-08 16:00:45 -07:00 |
|
Tamas Nemeth
|
2f79b50c24
|
fix(ingest): presto-on-hive - not failing on Hive type parsing error (#6118)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
|
2022-10-04 20:54:38 -07:00 |
|
Ravindra Lanka
|
055e4082da
|
fix(ingestion): fix percent change computation in stale_entity_removal (#6121)
|
2022-10-04 20:40:59 -07:00 |
|
Tamas Nemeth
|
3b9e9793a7
|
fix(ingest): bigquery-beta - handling complex types properly (#6062)
|
2022-09-27 21:31:24 +02:00 |
|
Harshal Sheth
|
3f1d47c069
|
feat(ingest): list referenced env variables in recipe (#6043)
|
2022-09-26 23:16:18 -07:00 |
|
Alexey Kravtsov
|
3c3ab64954
|
feat(ingest): implement compression for CheckpointState (#6007)
|
2022-09-26 10:18:42 -07:00 |
|
Harshal Sheth
|
27f28019de
|
refactor(ingest): move common host_port validation (#6009)
|
2022-09-22 16:32:07 -07:00 |
|
Ravindra Lanka
|
b8941ab190
|
feat(ingestion): Add fail-safe stale entity removal via configurable 'fail_safe_threshold' param. (#6027)
|
2022-09-22 16:09:22 -07:00 |
|
Harshal Sheth
|
68db859ca1
|
refactor(ingest): streamline two-tier db config validation (#5986)
|
2022-09-21 10:45:37 -07:00 |
|
Mayuri Nehate
|
b195b6c123
|
fix(ingest): encode reserved characters when creating dataset urn (#5977)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-09-20 16:59:02 -07:00 |
|
Harshal Sheth
|
937ab192c0
|
feat(ingest): add support for aliases in plugin registry (#5958)
|
2022-09-16 07:19:32 -07:00 |
|
skrydal
|
a026c84691
|
feat: qualifiedName support + populating glue ARN (#5952)
|
2022-09-15 21:15:03 -07:00 |
|
skrydal
|
f61a040555
|
feat(ingestion) Add more info to glue entities (#5874)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-09-14 12:25:09 -07:00 |
|
Harshal Sheth
|
e23523a781
|
fix(ingest): fix type annotations on some pydantic fields (#5795)
|
2022-09-14 11:05:31 -07:00 |
|
Harshal Sheth
|
a1e1d2fd0a
|
feat(ingest): add ConfigEnum type (#5734)
|
2022-09-14 09:57:42 -07:00 |
|
Ravindra Lanka
|
ee68f09624
|
feat(ingestion): Refactor standard state-handling tasks into a common handler that are common across all stateful ingestion sources. (#5766)
|
2022-09-14 09:30:42 -07:00 |
|
Mayuri Nehate
|
aedf1522fb
|
feat(ingest): snowflake-beta - minor changes, tests (#5910)
|
2022-09-12 10:42:52 -07:00 |
|
Harshal Sheth
|
e556bcb306
|
feat(ingest): add entity type inference to mcpw (#5880)
|
2022-09-10 20:36:10 -07:00 |
|
Harshal Sheth
|
220ae0b6c9
|
feat(ingest): make sink use type annotations (#5899)
|
2022-09-10 19:46:20 -07:00 |
|
Shirshanka Das
|
056add128d
|
fix(ingest): datahub-api - move instantiation to the right config class (#5878)
|
2022-09-09 13:34:21 -07:00 |
|
Harshal Sheth
|
6063484714
|
fix(ingest): avrogen handling for missing fields with default values (#5844)
|
2022-09-08 14:05:28 -07:00 |
|
Harshal Sheth
|
08622f25ef
|
feat(ingest): add utility for converting MCEs to MCPs (#5812)
|
2022-09-06 15:25:48 -07:00 |
|
mohdsiddique
|
2f65e2f226
|
feat(transformers): Add semantics & transform_aspect support in transformers (#5514)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
|
2022-09-06 14:44:14 -07:00 |
|
Shirshanka Das
|
9afda47085
|
feat(cli): add support for sampled reporting to keep logs manageable (#5800)
|
2022-09-01 14:47:28 -07:00 |
|
Harshal Sheth
|
c05f3970fd
|
feat(ingest): cli - add rewrite option for metadata file check (#5763)
|
2022-09-01 14:30:00 -07:00 |
|
Tamas Nemeth
|
4572c96d60
|
feat(ingestion): bigquery - Bigquery beta connector - first cut (#5663)
|
2022-08-30 07:33:24 +02:00 |
|
Harshal Sheth
|
eb87db9813
|
fix(ingest): proper null skip logic in serialization (#5749)
|
2022-08-29 16:34:58 -07:00 |
|