37 Commits

Author SHA1 Message Date
Harshal Sheth
f6764ee17a
chore(sdk): rename _schema_classes to mark it as internal-only (#13309) 2025-04-24 12:22:01 +05:30
Sergio Gómez Villamor
a0319af7db
fix(ingestion): fixes producing some URNs with reserved characters (#12772) 2025-03-05 14:25:20 +01:00
Harshal Sheth
12eb0cd1a7
feat(sdk): add search client (#12754) 2025-03-03 10:05:26 -08:00
Harshal Sheth
8dfd8fbbaf
feat(ingest): migrate Cassandra source to new SDK (#12695) 2025-02-24 11:57:51 -08:00
Harshal Sheth
282395dc4a
feat(sdk): support qualified urn names in simple urn constructors (#12426)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-01-22 13:46:24 -08:00
Harshal Sheth
5f63f3fba9
feat(sdk): support urns in other urn constructors (#12311) 2025-01-10 09:45:31 -08:00
Harshal Sheth
4e3103e266
fix(ingest): use typing_extensions.Self (#12230) 2024-12-27 10:50:43 -08:00
Harshal Sheth
7dbb3e60cb
chore(ingest): start using explicit exports (#11899) 2024-11-20 13:33:30 -08:00
Harshal Sheth
7f6f18cf77
feat(ingest): generate urn types for all entities (#11676) 2024-10-22 11:49:20 -07:00
Harshal Sheth
ea7d6a91e5
fix(ingest): improve auto_materialize_referenced_tags_terms error handling (#10906) 2024-07-12 22:32:54 -07:00
Harshal Sheth
20e15ffc58
feat(ingest): improve custom package metadata (#9985) 2024-03-11 14:14:59 -07:00
Harshal Sheth
4c83f10cdb
feat(ingest): throw codegen error on duplicate class names (#9960) 2024-02-29 16:13:43 -08:00
Harshal Sheth
d52f0305eb
fix(ingest): fix metadata for custom python packages (#9391) 2023-12-08 13:13:49 -05:00
Harshal Sheth
a7dc9c9d22
feat(sdk): autogenerate urn types (#9257) 2023-11-30 18:11:36 -05:00
Harshal Sheth
89dff8f7bd
chore(ingest): start working on pydantic v2 support (#9220) 2023-11-10 09:34:08 -08:00
Andrew Sikowitz
8a80e858a7
build(ingest): Bump avro pin: security vulnerability (#9042) 2023-10-25 13:06:12 -04:00
Harshal Sheth
791e2e7bf5
feat(python): support custom models without forking (#8774) 2023-09-22 16:43:58 -07:00
Harshal Sheth
a33153c1f6
feat(sdk): add DataHubGraph.get_entity_semityped method (#7905) 2023-04-26 13:44:13 -07:00
Harshal Sheth
af566e1184
feat(model): fully populate the entity registry (#7818) 2023-04-15 13:33:05 -07:00
Harshal Sheth
89734587f7
feat(ingest): add urn modification helper (#7440)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-16 13:27:08 -07:00
Harshal Sheth
f39ed5244c
fix(ingest): support multiple entity_registry sections (#7066) 2023-01-18 15:12:08 +05:30
Harshal Sheth
cb12910b6b
feat(ingest): add entity registry in codegen (#6984)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-17 19:41:43 -08:00
Harshal Sheth
0ca3383d30
feat(ingest): support reserved keywords in model codegen (#6351) 2022-11-02 22:26:56 -07:00
Harshal Sheth
8f72de5d8b
refactor(ingest): move aspect maps to dedicated file (#5821) 2022-09-10 18:43:11 -07:00
Harshal Sheth
41e8f7b203
feat(ingest): generate a list of aspects in codegen (#5633) 2022-08-15 17:21:40 -07:00
Harshal Sheth
9790f3cefa
feat(ingest): infer aspectName from aspect type in MCP (#5566) 2022-08-07 07:52:58 -07:00
Harshal Sheth
3c2a9c585b
fix(ingest): activate mypy support for ParamSpec typing annotation (#5551) 2022-08-04 17:05:52 +05:30
Vincent Koc
d60f789519
chore: Refactor Python Codebase (#5113) 2022-07-06 16:09:27 +05:30
Harshal Sheth
6e5d1fe42f
fix(ingest): switch to avro from deprecated avro-python3 (#3412) 2021-10-18 15:35:27 -07:00
Harshal Sheth
5d93f249b4
feat(ingest): expose additional types to Python via codegen (#2712) 2021-06-17 10:04:28 -07:00
Kevin Hu
24268c2021
feat(ingest): headers for codegen Python scripts (#2637) 2021-06-11 09:44:18 -07:00
Harshal Sheth
9f4de4b20a
fix(ingest): remove datahub.metadata import shortcut (#2449) 2021-04-30 21:10:12 -07:00
Harshal Sheth
2af4603e49
fix(ingest): enable mypy disallow_incomplete_defs and disallow_untyped_decorators (#2393) 2021-04-14 13:40:24 -07:00
Harshal Sheth
b8462028c3
feat(ingest): various minor fixes (#2246) 2021-03-17 23:05:05 -07:00
Harshal Sheth
e0560e27ba Start updating readme 2021-02-15 18:29:27 -08:00
Harshal Sheth
9332e6b878 Add isort to CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
c7892ada4c Codegen avro + datahub kafka sink (#3)
* Add codegen

* New architecture + setup file -> console pipeline

* Cleanup source loader

* Basic Kafka metadata source

* Kafka source and extractor

* Add kwargs construct interface

* Fix kafka source unit test

* start working on pipeline test

* kafka datahub sink

* Make myself a profile

* Ingest to datahub from kafka

* Update codegen

* Add restli transport

* Fix bug in restli conversion
2021-02-15 18:29:27 -08:00