12076 Commits

Author SHA1 Message Date
adriaanslechten
1295c44615
feat(ingest) LDAP groups ingestion (#2434) 2021-04-22 13:56:30 -07:00
Harshal Sheth
034c33a050
fix(ingest): use entrypoints lib instead of pkg_resources (#2438) 2021-04-22 00:13:47 -07:00
Gabe Lyons
c7b49de67b
feat(ingest): adding superset ingestion source (#2425) 2021-04-22 00:11:54 -07:00
Brian
3d6489fe97
fix(react): fix lineage sidebar buttons (#2433) 2021-04-21 15:15:52 -07:00
Harshal Sheth
56b66ed328
fix(ingest): support custom snowflake types (#2436) 2021-04-21 15:14:59 -07:00
Brian
1c071833ed
feat(react): add dataJob, dataFlow entity pages, refactor with fragments (#2419) 2021-04-21 12:06:31 -07:00
Fredrik Sannholm
e88a671959
Fix(search): fix datajob and dataflow search mappings (#2418) 2021-04-21 12:04:20 -07:00
Harshal Sheth
ffe49f061a
fix(ingest): fix chart type enum serialization and add tests for rest emitter (#2429) 2021-04-21 11:34:24 -07:00
Shirshanka Das
e602fd3a33
docs: Update agenda for Apr 23 townhall (#2431) 2021-04-21 10:18:22 -07:00
Harshal Sheth
79daec29b7
fix(ingest): ensure upstreams in airflow lineage emission are entities (#2427) 2021-04-20 20:44:38 -07:00
Harshal Sheth
7d1ec520e5
fix(ingest): include database info for snowflake (#2426) 2021-04-20 20:40:30 -07:00
Harshal Sheth
2fee45d127
feat: add s3 data platform and logo (#2424) 2021-04-20 16:40:51 -07:00
Brian
95b96572c5
feat(react): schema visualization add support for nested structs (#2423) 2021-04-20 14:00:10 -07:00
Brian
086c6ea705
fix(react): lineage sidebar buttons should refer to the selected entity(#2422) 2021-04-20 13:58:16 -07:00
Dexter Lee
c1c587d164
fix(kafka-setup): Fix start script for kafka setup (#2421) 2021-04-19 15:39:38 -07:00
Brian
7257fa81be
feat(react): update dataset entity default svg icon (#2417) 2021-04-19 11:58:04 -07:00
Thomas Larsson
7869a8f142
feature(ingestion): Adding the concept of transformers (#2411)
Fixes: #2410

Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
2021-04-18 11:15:05 -07:00
Dexter Lee
c46d842981
fix(k8s): Add credentials to kafka-setup job and clean up (#2415) 2021-04-18 11:07:19 -07:00
Harshal Sheth
27a672e697
feat(ingest): add Kafka-based emitter example (#2412) 2021-04-16 17:54:50 -07:00
Gabe Lyons
43075c4a91
fix(lineage): allow lineage viz to handle circular dependencies (#2413) 2021-04-16 17:53:01 -07:00
Dexter Lee
1b3a1459e3
fix(kafka-setup): Add the correct context to the git workflow for pushing kafka-setup image (#2414) 2021-04-16 17:51:43 -07:00
Harshal Sheth
9ac17c4ee0
fix(ingest): bump avro-gen3 (#2403)
Closes #2375.
2021-04-16 11:59:05 -07:00
Brian
5c88f7624a
feat(react): use default entity icon if lineageentity has no icon (#2406) 2021-04-16 09:46:02 -07:00
Harshal Sheth
91a2f69310
fix(ingest): properly handle fieldDiscriminator with restli (#2408) 2021-04-16 09:42:52 -07:00
Harshal Sheth
777c05973f
fix(ingest): add sqlalchemy extra (#2409) 2021-04-16 09:41:23 -07:00
G-nther
177161a850
feat(kafka-setup): add option for SSL and topic partition config via environment (#2398) 2021-04-16 09:40:39 -07:00
Dexter Lee
379c2e1c0c
feat(k8s): add extraEnvs to setup jobs (#2404) 2021-04-16 09:39:40 -07:00
Brian
ed3c3b236b
feat(react): add footer buttons in lineage sidebar (#2407) 2021-04-15 11:47:32 -07:00
Thomas Larsson
89fb538fa5
feature(ingestion): Make origin/fabric_type configurable (#2405)
Fixes: #2394

Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
2021-04-15 10:47:28 -07:00
Brian
82791008c3
feat(react): add padding between tags and description on datasets profile page (#2384) 2021-04-14 19:35:21 -07:00
Gabe Lyons
1e11851a90
feat(sample): adding sample mces for dataflows and datajobs (#2396) 2021-04-14 19:27:01 -07:00
Harshal Sheth
ffe03e6758
fix(ingest): streamline codegen init methods (#2400) 2021-04-14 19:25:57 -07:00
Brian
9a48e11b7b
feat(react): update schema table to have fixed description column, set line break with max description width (#2382) 2021-04-14 19:22:19 -07:00
Dexter Lee
f018dd6f3e
fix: Fix env variable setup for kafka, mysql-setup docker containers (#2402) 2021-04-14 19:07:50 -07:00
Harshal Sheth
9c2a30c3a1
fix(ingest): add db name to postgres URNs (#2401) 2021-04-14 15:04:13 -07:00
Harshal Sheth
2af4603e49
fix(ingest): enable mypy disallow_incomplete_defs and disallow_untyped_decorators (#2393) 2021-04-14 13:40:24 -07:00
Gabe Lyons
c57d5a3731
fix(react): fix access to pictureLink in charts and dashboards (#2395) 2021-04-14 13:23:53 -07:00
Gabe Lyons
a9f4f3797a
fix(tags): check description existence on tags (#2399) 2021-04-14 13:20:51 -07:00
Brian
6a5f065448
feat(react): fix long descriptions overflow issue in lineage side panel (#2383) 2021-04-13 18:21:41 -07:00
Harshal Sheth
a11329d5b8
refactor(ingest): update test harness to use a compose file per test (#2392) 2021-04-13 17:30:24 -07:00
Brian
dfc99d6b6a
feat(react): fix browse link of last breadcrumb linked to unknown page (#2391) 2021-04-13 14:22:43 -07:00
Dexter Lee
7c54004b93
feat(mysql-setup): Add the ability to specify database name for mysql-setup (#2385) 2021-04-13 08:03:50 -07:00
Harshal Sheth
fb6f74b1da
feat(ingest): add generic sqlalchemy source (#2389) 2021-04-13 08:01:38 -07:00
Dexter Lee
3ddf163a91
feat(k8s): Add ability to add service accounts to setup jobs (#2390) 2021-04-13 07:59:18 -07:00
Dexter Lee
fa015c5aaa
fix(kafka-topic-convention): Fix DAOs that do not refer to TopicConvention (#2387) 2021-04-13 07:58:31 -07:00
Dexter Lee
1fc532d831
feat(index): Add index naming convention for elasticsearch (#2386) 2021-04-13 07:56:31 -07:00
Harshal Sheth
eeee8aa34e
fix(ingest): report correct version status in dev mode (#2388) 2021-04-12 19:35:40 -07:00
Harshal Sheth
41cd52f9e2
feat(ingest): add Airflow lineage backend (#2368) 2021-04-12 17:40:15 -07:00
Junqiu Zheng
cadcdd936c
fix(docs): fix config example for file sink (#2380) 2021-04-12 11:37:17 -07:00
Dexter Lee
f293add5b1
feat(k8s): Update pods with correct probes and remove unnecessary dependencies (#2362) 2021-04-09 17:49:12 -07:00