3904 Commits

Author SHA1 Message Date
Harshal Sheth
6d875b8241
test(ingest): ensure transformer registry works for aliases (#2572) 2021-05-17 15:08:49 -07:00
Harshal Sheth
7af1a13138
fix(ingest): better active directory LDAP support (#2571) 2021-05-17 14:42:54 -07:00
Gary Lucas
af4f3b9683
fix(dbt): set target platform and load schema (#2483) 2021-05-17 12:22:52 -07:00
Albert Franzi
38e3f6d4d0
feat(ingest): add AWS IAM Roles Support to the Glue Source (#2563) 2021-05-17 12:19:34 -07:00
David Koenitzer
4958febed5
docs: Add Sphinx Docstrings to Airflow Modules (#2562) 2021-05-17 12:10:35 -07:00
Harshal Sheth
ebe7409897
fix(cli): prevent click from suppressing errors (#2560) 2021-05-17 11:50:38 -07:00
Harshal Sheth
f590f11ff3
fix(ingest): check mypy types for test helpers (#2561) 2021-05-17 11:42:12 -07:00
Harshal Sheth
3dfe3d375b
feat(ingest): add options for Airflow lineage backend (#2557) 2021-05-13 20:02:47 -07:00
Kevin Hu
5ab1cbbbb2
feat(ingest): MongoDB schema inference (#2546) 2021-05-13 19:44:33 -07:00
Fredrik Sannholm
133577557c
feat(ingest): Looker view and dashboard ingestion (#2493) 2021-05-13 11:42:53 -07:00
Harshal Sheth
e3c190e772
fix(ingest): register custom Hive types (#2543) 2021-05-12 17:54:59 -07:00
Harshal Sheth
8d910b9e39
docs(ingest): improve kafka schema registry config docs (#2544) 2021-05-12 17:54:33 -07:00
Harshal Sheth
a671001824
refactor(ingest): move Airflow into datahub_provider module (#2521) 2021-05-12 15:01:11 -07:00
Albert Franzi
7fce505ffb
feat(ingest): define Redshift as a Postgres Source (#2540) 2021-05-12 10:00:34 -07:00
Harshal Sheth
a47400f18e
build(ingest): use gradle in commands + docs (#2531) 2021-05-11 19:03:20 -07:00
Harshal Sheth
a2755cd00d
fix(ingest): remove mce.json file from root (#2536) 2021-05-11 18:51:51 -07:00
Brian
d7d8870008
feat(usergroup): implement corpgroup in graphql, refactor avatars and ownership in react (#2519) 2021-05-11 17:55:45 -07:00
Harshal Sheth
2811d23e45
feat(ingest): add a transformer for adding ownership (#2532) 2021-05-11 17:46:39 -07:00
Harshal Sheth
36922c1746
docs: enable better syntax highlighting (#2529) 2021-05-11 15:16:12 -07:00
Harshal Sheth
35841d2d85
feat(ingest): check in generated schema files (#2503) 2021-05-10 19:36:23 -07:00
Harshal Sheth
4b2a6bd46a
fix(ingest): generate Airflow tags correctly (#2522) 2021-05-10 19:26:55 -07:00
Harshal Sheth
0ddf0ab262
fix(ingest): add support for custom postgres types (#2524) 2021-05-10 18:06:56 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields (#2228)
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
2021-05-10 13:20:23 -07:00
Harshal Sheth
fe3ec3dee8
fix(build): only check for src/ and tests/ directories for lint checks (#2520) 2021-05-10 13:17:43 -07:00
Harshal Sheth
cd588baccb
build(ingest): include package data in sdist (#2513) 2021-05-07 15:21:43 -07:00
Harshal Sheth
d0ca3191c9
build(ingest): add metadata-ingestion to gradle build (#2510) 2021-05-06 22:10:49 -07:00
Harshal Sheth
50ea58c32d
docs: improve airflow explanations and examples (#2509) 2021-05-06 19:12:19 -07:00
Harshal Sheth
0e8b3d97ab
fix(ingest): remove double edges from Airflow lineage backend (#2508) 2021-05-06 19:08:02 -07:00
Vincenzo Lavorini
3132910707
docs: fixed MCE file recipe example (#2505) 2021-05-06 19:05:57 -07:00
Harshal Sheth
3edbdcb3e9
ci(ingest): ensure datahub imports work (#2502) 2021-05-06 07:48:56 -07:00
Harshal Sheth
1facfbd5a3
feat(ingest): capture table properties if available (#2497) 2021-05-05 14:07:08 -07:00
Harshal Sheth
7f0656fd5e
fix(ingest): replace ImportError with ModuleNotFoundError (#2498)
Using the more specific exception will prevent us from accidentally
ignoring errors that should be handled.
2021-05-05 14:05:16 -07:00
Harshal Sheth
c32bf494d5
fix(ingest): support https connections with cookies in Hive ingestion (#2489)
Tested locally.
2021-05-04 13:10:52 -07:00
Harshal Sheth
6f1f0a4845
feat(ingest): support hive over http (#2486) 2021-05-03 22:11:50 -07:00
Harshal Sheth
9f4de4b20a
fix(ingest): remove datahub.metadata import shortcut (#2449) 2021-04-30 21:10:12 -07:00
Harshal Sheth
90108f4aba
docs(ingest): add gradle build step to developing setup as well (#2479) 2021-04-30 21:07:51 -07:00
Harshal Sheth
71933a9f31
test(ingest): rename TestSource -> FakeSource (#2481) 2021-04-30 20:54:07 -07:00
Harshal Sheth
201ffd4979
test: add smoke test (#2464) 2021-04-29 23:27:03 -07:00
Harshal Sheth
df9e7c594f
fix(ingest): guess hook type from name (#2475) 2021-04-29 23:23:19 -07:00
Harshal Sheth
e48a74b80a
test(ingest): add test names and IDs using pytest (#2476) 2021-04-29 23:18:55 -07:00
Harshal Sheth
5553dc820c
fix(ingest): use postgres data platform urn (#2472) 2021-04-28 11:34:45 -07:00
Rickard Cardell
48ea19a5f7
feat(react): add custom properties tab on dashboard profile page (#2439) (#2469) 2021-04-27 11:26:09 -07:00
Brian
5894365d77
fix(bootstrap): update bootstrap data with more realistic nested schemas (#2470) 2021-04-27 11:19:40 -07:00
Gabe Lyons
3bf5ffab5d
docs(ingest): Update README.md to add superset source (#2466) 2021-04-26 23:10:27 -07:00
Harshal Sheth
50aee5c05a
fix(ingest): support Airflow 1.10.x style lineage in Airflow 2 (#2455) 2021-04-26 23:08:43 -07:00
Harshal Sheth
694c4ce8ec
test(ingest): fix mypy issue in schema util test (#2465) 2021-04-26 18:18:37 -07:00
Harshal Sheth
83fdc6417f
feat(ingest): capture default values in Avro schemas (#2463) 2021-04-26 17:07:29 -07:00
Harshal Sheth
d415234a8c
fix(ingest): fields with defaults should be optional (#2461) 2021-04-26 16:45:48 -07:00
Harshal Sheth
2da5e1fd10
feat(ingest): setup scaffolding for tox testing (#2451) 2021-04-26 16:44:36 -07:00
Harshal Sheth
e075acaec9
docs(ingest): clarify options field and fix bigquery sample config (#2456) 2021-04-26 16:30:05 -07:00