Mayuri Nehate
84270bcac8
feat(ingest/nifi): kerberos authentication ( #8097 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
Co-authored-by: Indy Prentice <iprentic@users.noreply.github.com>
2023-05-24 15:09:01 -07:00
Harshal Sheth
4873a32e4a
fix(ingest): emitter bug fixes ( #8093 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-05-23 12:04:16 -07:00
Shirshanka Das
b3c790aab6
feat: Add support for Data Products ( #8039 )
...
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2023-05-17 07:17:25 +00:00
Harshal Sheth
1ebc88caf2
fix: build vercel python from source ( #7972 )
2023-05-05 17:10:05 +02:00
Harshal Sheth
a33153c1f6
feat(sdk): add DataHubGraph.get_entity_semityped method ( #7905 )
2023-04-26 13:44:13 -07:00
Harshal Sheth
af566e1184
feat(model): fully populate the entity registry ( #7818 )
2023-04-15 13:33:05 -07:00
Harshal Sheth
342830c68c
fix(cli): use correct ingestion image in script ( #7826 )
2023-04-14 23:47:08 -07:00
Harshal Sheth
82dc2b6393
feat(docs): clear up source configs ( #7720 )
2023-04-04 18:40:19 +05:30
Harshal Sheth
575909e41c
feat(docs): support inlining code snippets from files ( #7712 )
2023-03-30 00:02:21 +00:00
Harshal Sheth
d9231addd2
feat(docs-website): add vercel preview environment ( #7644 )
2023-03-22 12:28:52 -07:00
Shirshanka Das
104c9811f5
fix(ingest/docs): improve matcher to include types with spaces in them ( #7631 )
2023-03-18 12:59:43 -07:00
Shirshanka Das
41d4c0b074
feat(ingest/docs): json-schema fixes, improvements to ingestion doc generation ( #7615 )
2023-03-17 15:58:14 +01:00
Harshal Sheth
89734587f7
feat(ingest): add urn modification helper ( #7440 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-16 13:27:08 -07:00
Harsha Mandadi
bf36c935fa
feat(ingest/s3): support path_specs of different S3 buckets in the same recipe ( #7514 )
2023-03-14 21:55:57 -07:00
Andrew Sikowitz
8fd2cc5f20
fix(ingest/snowflake): Improve memory usage of metadata extraction ( #7349 )
2023-02-20 14:46:10 +01:00
Felix Lüdin
da2b0c9e1b
fix(docs): sort sources by display name in doc's sidebar ( #7322 )
2023-02-13 12:39:54 -08:00
Harshal Sheth
bc9adfa441
ci(ingest): add flag to skip tests but run codegen during release ( #7067 )
2023-01-18 11:55:07 +01:00
Harshal Sheth
f39ed5244c
fix(ingest): support multiple entity_registry sections ( #7066 )
2023-01-18 15:12:08 +05:30
Harshal Sheth
cb12910b6b
feat(ingest): add entity registry in codegen ( #6984 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-17 19:41:43 -08:00
Harshal Sheth
62a2aa94f6
feat: remove jq requirement + tweak modeldocgen args ( #6904 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-30 14:02:57 -05:00
Harshal Sheth
dfc5c6bfce
chore(ingest): remove inferred args to MCPW, part 1 ( #6819 )
2022-12-30 01:26:47 -05:00
Harshal Sheth
7d63399d00
fix(ingest): fix serde for empty dicts in unions with null ( #6745 )
...
The code changes in https://github.com/acryldata/avro_gen/pull/16 , but tests are written here.
2022-12-13 08:17:24 +01:00
Harshal Sheth
3e907ab0d1
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ ( #6204 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-11 15:04:36 -05:00
Harshal Sheth
0ca3383d30
feat(ingest): support reserved keywords in model codegen ( #6351 )
2022-11-02 22:26:56 -07:00
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x ( #6331 )
2022-11-01 21:12:34 -07:00
Ravindra Lanka
b638bcf442
feat(ingestion): Documentation on adding stateful ingestion use-cases to new sources ( #5985 )
2022-09-21 09:02:50 -07:00
Shirshanka Das
f70bb201ce
fix(build): fix preflight script for m1 for lib postgresql ( #5957 )
2022-09-15 13:21:02 -07:00
Harshal Sheth
e23523a781
fix(ingest): fix type annotations on some pydantic fields ( #5795 )
2022-09-14 11:05:31 -07:00
Harshal Sheth
a1e1d2fd0a
feat(ingest): add ConfigEnum type ( #5734 )
2022-09-14 09:57:42 -07:00
Harshal Sheth
8f72de5d8b
refactor(ingest): move aspect maps to dedicated file ( #5821 )
2022-09-10 18:43:11 -07:00
Shirshanka Das
b9068ffd2e
fix(m1): preflight works with brew postgres@14 ( #5868 )
2022-09-08 15:02:30 +05:30
Shirshanka Das
d911b15da5
docs(ingest): improve doc gen, docs for snowflake, looker ( #5867 )
2022-09-08 00:38:10 -07:00
Harshal Sheth
abddc01877
fix(ingest): fix doc generation import ordering issue with postgres ( #5846 )
...
Relying on the correct import directly, rather than going through
SQLAlchemy's import wrapper (in their dialect.py) allows us to bypass
this strange error in doc generation.
2022-09-07 21:44:33 +05:30
Harshal Sheth
6d83eb9449
refactor(ingest): prefer as imports instead of pegasus2avro files ( #5820 )
2022-09-05 21:09:07 -07:00
Shirshanka Das
fe52df6bab
fix(docs): fixes issue with auto-generated ingestion doc ( #5733 )
2022-08-26 09:31:53 -07:00
Harshal Sheth
41e8f7b203
feat(ingest): generate a list of aspects in codegen ( #5633 )
2022-08-15 17:21:40 -07:00
Harshal Sheth
9790f3cefa
feat(ingest): infer aspectName from aspect type in MCP ( #5566 )
2022-08-07 07:52:58 -07:00
Piotr Sierkin
828a711684
feat(ingest): dbt - control over emitting test_results, test_definitions, etc. ( #5328 )
...
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-06 21:42:53 -07:00
Harshal Sheth
3c2a9c585b
fix(ingest): activate mypy support for ParamSpec typing annotation ( #5551 )
2022-08-04 17:05:52 +05:30
Tamas Nemeth
d08a8f79ab
feat(ingest): preflight - Add way to check/upgrade brew package version in preflight if needed ( #5435 )
2022-07-22 14:20:12 +05:30
Vincent Koc
d60f789519
chore: Refactor Python Codebase ( #5113 )
2022-07-06 16:09:27 +05:30
Shirshanka Das
64fae6121a
feat(docs): ingest - add tab for config json schema ( #5086 )
2022-06-03 14:57:36 -07:00
Shirshanka Das
b2d957ddbf
feat(docs): make config section for ingestion sources auto-open by default ( #5075 )
2022-06-02 18:27:13 +02:00
Shirshanka Das
83d9f7d6e6
fix(ingest): fix build breakage due to traitlets 5.2.2 bug ( #5051 )
2022-05-31 14:42:56 -07:00
Shirshanka Das
3bb8a41c2a
fix(build): docgen should fail if plugin is not loadable ( #5038 )
2022-05-29 08:09:02 -07:00
Aseem Bansal
21f8d4be03
fix(redash): improve logging for debugging, add validation for dataset urn, some refactoring ( #4997 )
2022-05-26 16:24:54 +05:30
Aseem Bansal
b77f381429
doc(ingestion): default boolean fix, broken bigquery docgen ( #4984 )
2022-05-24 23:17:27 +05:30
Shirshanka Das
8b235b1706
fix(docs): ingest - sort modules, fix small typos ( #4880 )
2022-05-09 13:01:44 -07:00
Shirshanka Das
d152ca367c
fix(docgen): fix failure count incrementing during doc generation ( #4806 )
2022-05-03 22:56:35 +02:00
Shirshanka Das
a9ad138172
feat(ingest): docs - overhaul source connector docs to make it code driven ( #4798 )
...
Co-authored-by: MugdhaHardikar-GSLab <mugdha.hardikar@gslab.com>
2022-05-02 00:18:15 -07:00
