27 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
1563b0e9fb
fix(ingestion): use default generate_browse_path_v2 even if no pipeline_config (#13117) 2025-04-23 13:25:58 +02:00
skrydal
38f1553315
feat(ingestion): Refactoring timestamping logic for WorkUnits + custom logic for Iceberg (#13030)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-04 22:30:27 +02:00
Harshal Sheth
93c8ae2267
fix(ingest/snowflake): handle dots in snowflake table names (#12105) 2024-12-12 15:31:32 +05:30
Tamas Nemeth
16a02411c3
fix(ingest/sagemaker): Gracefully handle missing model group (#12000) 2024-12-03 10:48:04 +01:00
Harshal Sheth
b8144699fd
chore(ingest): reorganize unit tests (#11636) 2024-10-16 19:18:32 -07:00
sagar-salvi-apptware
50ed448861
fix(ingest/sagemaker): ensure consistent STS token usage with refresh mechanism (#11170)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-08-22 15:42:13 +05:30
Tony Ouyang
a5515c5d47
feat(ingestion/SageMaker): Remove deprecated apis and add stateful ingestion capability (#10573) 2024-05-28 12:16:28 +02:00
Andrew Sikowitz
bf9f380350
fix(ingest): Generate browse paths v2 for more sources; properly pass platform_instance (#8501) 2023-07-25 11:35:34 +05:30
Harshal Sheth
667ca8632d
feat(ingest): avoid embedding serialized json in metadata files (#6742) 2022-12-28 19:28:38 -05:00
John Joyce
2e3a25123d
refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker (#6002) 2022-12-21 08:02:59 -08:00
Tamas Nemeth
e41b455e14
fix(ingest): bigquery - sharded table support improvements (#6789) 2022-12-19 18:57:37 +01:00
Kevin Hu
02fe05eb8f
feat(ingest): data-lake - remove spark requirement if not profiling (#4131) 2022-02-24 23:26:06 -08:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect (#2983)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960) 2021-07-29 20:04:40 -07:00
Gabe Lyons
d9d03c9bd9
feat(ml-model): adding ml models and ml model groups (#2961) 2021-07-28 20:39:05 -07:00
Kevin Hu
bf2775f6cf
feat(ingest): type stubs for boto3 (#2975) 2021-07-28 20:35:35 -07:00
Kevin Hu
662017ef17
fix(ingest): patch lookml types and refactor ingestion sources layout (#2950) 2021-07-26 13:06:52 -07:00
Kevin Hu
43e61d9628
feat(models): remove versions from metrics and hyperparams (#2938) 2021-07-22 22:18:45 -07:00
Kevin Hu
757abfc6f6
fix(ingest): fix browsepaths and ownership urns (#2935) 2021-07-22 15:26:10 -07:00
Kevin Hu
6dbc59940a
feat(ingest): refactor mlModel grouping and add browsepaths (#2929) 2021-07-22 13:33:15 -07:00
Harshal Sheth
7dc85a478e
feat(ingest): add make_data_platform_urn method to builder (#2926) 2021-07-22 13:25:07 -07:00
Kevin Hu
736249f0c7
feat(ingest): extract SageMaker metrics, hyperparameters, and external URLs (#2910) 2021-07-21 21:30:07 -07:00
Kevin Hu
6abd5e191a
feat(ingest): lineage for SageMaker model endpoints and groups (#2894) 2021-07-19 11:30:43 -07:00
Kevin Hu
44ed2f3684
feat(ingest): extract lineage between SageMaker jobs and models (#2868) 2021-07-15 18:56:13 -07:00
Kevin Hu
a2106ca9e8
feat(ingest): SageMaker jobs and models (#2830) 2021-07-08 16:16:16 -07:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option (#2812) 2021-06-30 16:53:20 -07:00
Kevin Hu
4da76726d3
feat(ingest): SageMaker feature store ingestion (#2758) 2021-06-29 19:43:31 -07:00