1604 Commits

Author SHA1 Message Date
Harshal Sheth
c32ba3c5ef
feat(ingest): add timezone validation to bigquery usage (#2962) 2021-07-28 14:16:31 -07:00
Harshal Sheth
fa22b8e17b
feat(ingest): add Airflow TaskFlow example (#2958) 2021-07-26 13:09:25 -07:00
Kevin Hu
662017ef17
fix(ingest): patch lookml types and refactor ingestion sources layout (#2950) 2021-07-26 13:06:52 -07:00
Kevin Hu
f82ea3abef
feat(ingest): ingest descriptions from dbt models (#2955) 2021-07-26 10:47:25 -07:00
Harshal Sheth
6135eed757
test(ingestion): fix flaky package discovery test (#2949) 2021-07-25 21:15:22 -07:00
Harshal Sheth
7a1ce19d2b
fix(ingestion): resolve test bugs for 3.6 (#2953) 2021-07-23 17:07:13 -07:00
Kevin Hu
43e61d9628
feat(models): remove versions from metrics and hyperparams (#2938) 2021-07-22 22:18:45 -07:00
Harshal Sheth
90e05df791
fix(ingest): refactor + fix recursion in lookml file loading logic (#2913) 2021-07-22 15:54:01 -07:00
Harshal Sheth
6e833e761e
feat(ingest): improve bigquery-usage robustness and docs (#2925) 2021-07-22 15:37:55 -07:00
Kevin Hu
757abfc6f6
fix(ingest): fix browsepaths and ownership urns (#2935) 2021-07-22 15:26:10 -07:00
Kevin Hu
6dbc59940a
feat(ingest): refactor mlModel grouping and add browsepaths (#2929) 2021-07-22 13:33:15 -07:00
Harshal Sheth
7dc85a478e
feat(ingest): add make_data_platform_urn method to builder (#2926) 2021-07-22 13:25:07 -07:00
Kevin Hu
84a1486b89
feat(ingest): dbt aliases (#2927) 2021-07-22 07:57:14 -07:00
Kevin Hu
736249f0c7
feat(ingest): extract SageMaker metrics, hyperparameters, and external URLs (#2910) 2021-07-21 21:30:07 -07:00
Kevin Hu
d6875b4f4f
feat(ingest): test dbt ingestion with and without schemas (#2922) 2021-07-21 12:56:31 -07:00
Harshal Sheth
ad30f2b8ec
feat(ingestion): support multiple project IDs in bigquery usage stats (#2920) 2021-07-21 12:42:06 -07:00
Harshal Sheth
5f0b4464f5
fix(ingest): pin snowflake sqlalchemy connector (#2923) 2021-07-20 19:28:40 -07:00
Harshal Sheth
416f2a95df
feat(ingest): add support for Oracle spatial types (#2909) 2021-07-20 19:13:49 -07:00
Kevin Hu
859fab8bd3
docs(ingest): update sample recipe and test input for dbt (#2917) 2021-07-20 13:28:37 -07:00
aseembansal-gogo
6e1b2cf4f7
feat(ingest): Add option to change name of database for postgres (#2898) 2021-07-20 07:01:42 -07:00
Kevin Hu
6abd5e191a
feat(ingest): lineage for SageMaker model endpoints and groups (#2894) 2021-07-19 11:30:43 -07:00
Harshal Sheth
45e931dbac
feat(ingest): add can_add_aspect method for MCEs (#2905) 2021-07-17 20:02:07 -07:00
Harshal Sheth
89bdfdf1a8
fix(ingest): correct globs in lookml model discovery (#2891) 2021-07-16 11:59:50 -07:00
Kevin Hu
44ed2f3684
feat(ingest): extract lineage between SageMaker jobs and models (#2868) 2021-07-15 18:56:13 -07:00
Harshal Sheth
8e573fdb31
fix(ingest): fix druid misconfiguration bug (#2882) 2021-07-14 20:29:23 -07:00
Harshal Sheth
fe6bfc9685
fix(ingest): default to unlimited query log delay in bigquery-usage (#2881) 2021-07-14 20:05:31 -07:00
Harshal Sheth
be39037b10
build(ingest): reduce dependencies for dev install (#2872) 2021-07-14 20:02:48 -07:00
Kevin Hu
904d4410fe
feat(ingest): update golden files only when diff fails (#2869) 2021-07-13 14:59:22 -07:00
Kevin Hu
bc84c82a68
feat(ingest): extract dbt meta fields (#2876) 2021-07-13 14:58:25 -07:00
Kevin Hu
c4e2b9afa2
feat(ingest): add browse paths + dataplatform for Feast features (#2849) 2021-07-12 11:05:18 -07:00
Harshal Sheth
220dfe728c
feat(ingest): support dynamic imports for transfomer methods (#2858) 2021-07-12 11:03:53 -07:00
Kevin Hu
a2106ca9e8
feat(ingest): SageMaker jobs and models (#2830) 2021-07-08 16:16:16 -07:00
Kevin Hu
799b0634e1
fix(ingest): check for dbt materialization before proceeding (#2842) 2021-07-08 15:12:37 -07:00
Harshal Sheth
2f921d15e8
fix(ingest): avoid setting timestamps unless source system provides it (#2843) 2021-07-08 12:11:06 -07:00
Harshal Sheth
6fe663bf6a
feat(ingest): basic support for complex hive types (#2804) 2021-06-30 22:57:13 -07:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option (#2812) 2021-06-30 16:53:20 -07:00
Harshal Sheth
e51f86a9de
feat(ingest): support ingesting from multiple snowflake dbs (#2793) 2021-06-30 15:54:17 -07:00
Kevin Hu
4da76726d3
feat(ingest): SageMaker feature store ingestion (#2758) 2021-06-29 19:43:31 -07:00
Harshal Sheth
79f60d8b8a
refactor(ingest): remove deprecated methods and warn on deprecated import (#2797) 2021-06-29 11:43:43 -07:00
Harshal Sheth
5e69a4355e
refactor(ingest): use common get_sys_time method (#2782) 2021-06-28 20:40:10 -07:00
Harshal Sheth
19b2a42a00
feat: usage stats (part 2) (#2762)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-06-24 19:44:59 -07:00
Harshal Sheth
937f02c6bc
feat: usage stats (part 1) (#2750)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-06-24 17:11:00 -07:00
Kevin Hu
22a2ed81e4
feat(ingest): ingest last-modified from dbt sources.json (#2729) 2021-06-23 13:56:20 -07:00
Kevin Hu
96dde2c734
fix(ci): increase wait-for-it timeout to fix flaky feast test (#2747) 2021-06-23 11:37:31 -07:00
Kevin Hu
fffc61794c
feat(ingest): print docker logs on timeout (#2743) 2021-06-22 17:05:08 -07:00
Kevin Hu
4ddc4c28be
fix(ingest): fix lookml platform URN (#2742) 2021-06-22 13:55:29 -07:00
Kevin Hu
a89094da5b
feat(ingest): add support for Glue ETL jobs (#2687) 2021-06-22 11:33:22 -07:00
Kevin Hu
554e1637c5
fix(ingest): types for dbt (#2716) 2021-06-22 10:37:08 -07:00
Kevin Hu
2c20bce79d
fix(ci): increase Feast docker setup timeout (#2722) 2021-06-22 10:34:01 -07:00
Harshal Sheth
5d93f249b4
feat(ingest): expose additional types to Python via codegen (#2712) 2021-06-17 10:04:28 -07:00