3985 Commits

Author SHA1 Message Date
Harshal Sheth
288d17f07e
docs(ingest): update links to Kafka docs (#2834) 2021-07-06 15:33:52 -07:00
Harshal Sheth
2a9fdfc181
fix(cli): change docker nuke to also remove stopped containers (#2825) 2021-07-02 20:27:28 -07:00
Harshal Sheth
1b4788537d
feat(ci): separate metadata-ingestion into a separate workflow (#2828) 2021-07-02 20:26:00 -07:00
Harshal Sheth
6ef0bf0dfd
fix(ingest): convert superset timestamps to micros (#2827)
* Previous values were in seconds, which rendered incorrectly in the UI.
2021-07-02 20:21:56 -07:00
Gabe Lyons
8d6362419f
docs(elastic-for-graph): Add migrating from neo4j to elastic instructions (#2826) 2021-07-02 20:19:33 -07:00
Harshal Sheth
4c39d86f63
docs(ingest): add extra info for Redshift behind a proxy (#2817) 2021-07-02 10:31:14 -07:00
Harshal Sheth
d6c3043bc4
fix(ingest): view handling resilience for redshift (#2816) 2021-07-02 10:26:50 -07:00
Harshal Sheth
6fe663bf6a
feat(ingest): basic support for complex hive types (#2804) 2021-06-30 22:57:13 -07:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option (#2812) 2021-06-30 16:53:20 -07:00
Harshal Sheth
e51f86a9de
feat(ingest): support ingesting from multiple snowflake dbs (#2793) 2021-06-30 15:54:17 -07:00
Harshal Sheth
906ad67a37
refactor(ingest): extract sqlalchemy uri generation logic (#2786) 2021-06-30 09:33:41 -07:00
Harshal Sheth
ef75bc6e5c
fix(ingest): better warnings and error handling for rest sink (#2800) 2021-06-29 22:37:44 -07:00
Kevin Hu
4da76726d3
feat(ingest): SageMaker feature store ingestion (#2758) 2021-06-29 19:43:31 -07:00
Remi
2aa95ec750
feat(ingest): Improve lookml sql derived tables detection, add cascading derived tables to lineage (#2770) 2021-06-29 19:41:34 -07:00
Harshal Sheth
c8fe8d4026
fix(ingest): quote table names in hive (#2801) 2021-06-29 17:51:01 -07:00
Harshal Sheth
57ddb41086
fix(ingest): handle case when view definition handler is not implemented (#2796) 2021-06-29 17:49:57 -07:00
Harshal Sheth
79f60d8b8a
refactor(ingest): remove deprecated methods and warn on deprecated import (#2797) 2021-06-29 11:43:43 -07:00
Kevin Hu
7083de9b4f
fix(ingest): use correct platform for MongoDB ingestion (#2783) 2021-06-29 10:16:58 -07:00
Kevin Hu
14294e8f89
fix(docs): links to Feast entities (#2780) 2021-06-29 08:12:11 -07:00
Harshal Sheth
5e69a4355e
refactor(ingest): use common get_sys_time method (#2782) 2021-06-28 20:40:10 -07:00
Kevin Hu
09bbcea0a8
feat(ingest): add non-random sampling for mongo (#2778) 2021-06-27 23:40:17 -07:00
Harshal Sheth
c05459b446
docs: upgrade docusaurus, minor ingestion updates (#2774) 2021-06-27 23:38:38 -07:00
Harshal Sheth
424139145b
docs(ingest): move usage stats docs into the "sources" section (#2766) 2021-06-24 23:03:26 -07:00
Harshal Sheth
19b2a42a00
feat: usage stats (part 2) (#2762)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-06-24 19:44:59 -07:00
Harshal Sheth
937f02c6bc
feat: usage stats (part 1) (#2750)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-06-24 17:11:00 -07:00
Gabe Lyons
62ba937bb7
feat(elastic-as-graph): defaulting to elastic in quickstart (#2753) 2021-06-24 15:44:03 -07:00
Harshal Sheth
82468016ae
fix(docker): use head tag for datahub-ingestion (#2760) 2021-06-24 13:16:49 -07:00
Remi
91f5d4f59a
feat(ingest): add option to specify source platform database in lookml ingestion (#2749) 2021-06-23 16:16:20 -07:00
Kevin Hu
22a2ed81e4
feat(ingest): ingest last-modified from dbt sources.json (#2729) 2021-06-23 13:56:20 -07:00
John Joyce
8fc194717c
feat(quickstart): remove orphaned docker containers on quickstart through cli (#2748) 2021-06-23 13:46:19 -07:00
Kevin Hu
96dde2c734
fix(ci): increase wait-for-it timeout to fix flaky feast test (#2747) 2021-06-23 11:37:31 -07:00
Kevin Hu
fffc61794c
feat(ingest): print docker logs on timeout (#2743) 2021-06-22 17:05:08 -07:00
Kevin Hu
4ddc4c28be
fix(ingest): fix lookml platform URN (#2742) 2021-06-22 13:55:29 -07:00
Kevin Hu
a89094da5b
feat(ingest): add support for Glue ETL jobs (#2687) 2021-06-22 11:33:22 -07:00
Kevin Hu
554e1637c5
fix(ingest): types for dbt (#2716) 2021-06-22 10:37:08 -07:00
Kevin Hu
2c20bce79d
fix(ci): increase Feast docker setup timeout (#2722) 2021-06-22 10:34:01 -07:00
Remi
4944143042
fix(looker): fix invalid URN syntax error (#2737) 2021-06-22 10:05:17 -07:00
Gabe Lyons
0750332714
fix(editable descriptions): adding indexing for editable descriptions (#2710) 2021-06-17 10:55:26 -07:00
vijayan-nallasami-curve
26329e0af9
added node_type_pattern in dbt yaml file (#2705) 2021-06-17 10:24:33 -07:00
vijayan-nallasami-curve
943c54a835
Removed key not in catalog filter from dbt and added node filter using AllowDenyPattern (#2702) 2021-06-17 10:24:12 -07:00
Harshal Sheth
5d93f249b4
feat(ingest): expose additional types to Python via codegen (#2712) 2021-06-17 10:04:28 -07:00
Harshal Sheth
7e9a04479b
test(ingest): simplify docker cleanup commands (#2699) 2021-06-16 16:59:28 -07:00
Harshal Sheth
26dcece8ec
fix(ingest): use looker data platform (#2708) 2021-06-16 16:58:13 -07:00
Kevin Hu
63fe82995b
feat(ingest): Add test case and docs for SQL view ingestion (#2709) 2021-06-16 16:51:57 -07:00
Brian
a5f9b8dfe9
feat(entities): add markdown description update/viewer feature in dataset, datajob, dataflow, chart and dashboard, update ui/ux (#2707) 2021-06-16 15:48:27 -07:00
Gabe Lyons
523c3bf1d4
feat(aspects): support fetching of versioned aspects (#2677) 2021-06-16 10:03:21 -07:00
John Joyce
cc95916201
feat(gms): Merge MAE, MCE consumers into GMS (#2690) 2021-06-15 08:44:15 -07:00
John Joyce
7591c8994a
feat(datahub cli): DataHub CLI Quickstart (#2689) 2021-06-14 17:15:24 -07:00
Kevin Hu
1c364b52f6
feat(docs): Docs for S3 ingestion with AWS Glue (#2672) 2021-06-14 17:08:50 -07:00
Remi
9fb2e6aaa8
Only work with dbt catalog data if load_catalog is False (#2686) 2021-06-14 16:58:19 -07:00