mirror of
https://github.com/datahub-project/datahub.git
synced 2025-06-27 05:03:31 +00:00
24 KiB
24 KiB
Town Hall History
:::note For the Town Hall meetings after June 2023, please refer to our LinkedIn Live event history. :::
June 2023
- Community & Project Updates - Maggie Hays & Shirshanka Das (DataHub)
- Community Case Study: Dataset Joins - Raj Tekal & Bobbie-Jean Nowak (Optum)
- DataHub 201: Column-Level Data Lineage - Hyejin Yoon (DataHub)
- Sneak Peek: BigQuery Column-Level Lineage with SQL Parsing - Harshal Sheth (DataHub)
- DataHub Performance Tuning – Indy Prentice (DataHub)
May 2023
Agenda
- Community - Maggie Hays & Shirshanka Das (DataHub)
- Community Case Study: Jira + DataHub for Access Requests - Joshua Garza (Sharp Healthcare)
- Sneak Peek: Use your own ownership types - Pedro Silva (DataHub)
- Sneak Peek: Data Contracts are coming! – John Joyce, Shirshanka (DataHub)
- Bring DataHub into your BI Tools — Chris Collins (DataHub)
Apr 2023
Agenda
- Community & Roadmap Updates - Maggie Hays & Shirshanka Das (DataHub)
- DataHub 201: Python SDK - Hyejin Yoon (DataHub)
- Streamlined Search & Browse Experience - Chris Collins (DataHub)
- DataHub GitHub Actions - Harshal Sheth (DataHub)
- Data Products in DataHub - Shirshanka Das & Chris Collins (DataHub)
- DataHub Docs Bot - Maggie Hays (DataHub)
Mar 2023
Agenda
- Community & Roadmap Update
- Recent Releases
- Community Case Study — Jumio’s DataHub adoption journey
- DataHub 201: Data Debugging
- Sneak Peek: Streamlined Filtering Experience
Feb 2023
Agenda
- Community & Roadmap Update
- Recent Releases
- Community Case Study - How the Hurb Team successfully implemented and adopted DataHub within their organization
- Sneak Peek: Subscriptions and Notifications
- Search Improvements - API support for pagination
- New Feature - Custom Queries
- Simplifying Metadata Ingestion
- DataHub 201: Rolling Out DataHub
Jan 2023 (26th)
Agenda
- What’s to Come - Q1 2023 Roadmap: Data Products, Data Contracts and more
- Community Case Study - Notion: Automating annotations and metadata propagation
- Community Contribution - Grab: Improvements to documentation editing
- Simplifying DataHub - Removing Schema Registry requirement and introducing DataHub Lite
Jan 2023 (5th)
Agenda
- DataHub Community: 2022 in Review - Our Community of Data Practitioners is one of a kind. We’ll take the time to celebrate who we are, what we’ve built, and how we’ve collaborated in the past 12 months.
- Search Improvements - Learn how we’re making the Search experience smarter and faster to connect you with the most relevant resources during data discovery.
- Removing Schema Registry Requirement - Hear all about ongoing work to simplify the DataHub deployment process.
- Smart Data Profiling - We’re making big improvements to data profiling! Smart data profiling will reduce processing time by only scanning datasets that have recently changed.
- Sneak Peek: Time-based Lineage - Get a preview of how you’ll soon be able to trace lineage between datasets across different points in time to understand how interdependencies have evolved.
- Sneak Peek: Chrome Extension - Soon, you’ll be able to quickly access rich metadata from DataHub while exploring resources in Looker via our upcoming Chrome Extension.
Dec 2023
Agenda
November Town Hall (in December!)
- Community Case Study - The Pinterest Team will share how they have integrated DataHub + Thrift and extended the Metadata Model with a Data Element entity to capture semantic types.
- NEW! Ingestion Quickstart Guides - DataHub newbies, this one is for you! We’re rolling out ingestion quickstart guides to help you quickly get up and running with DataHub + Snowflake, BigQuery, and more!
- NEW! In-App Product Tours - We’re making it easier than ever for end-users to get familiar with all that DataHub has to offer - hear all about the in-product onboarding resources we’re rolling out soon!
- DataHub UI Navigation and Performance - Learn all about upcoming changes to our user experience to make it easier (and faster!) for end users to work within DataHub.
- Sneak Peek! Manual Lineage via the UI - The Community asked and we’re delivering! Soon you’ll be able to manually add lineage connections between Entities in DataHub.
- NEW! Slack + Microsoft Teams Integrations - Send automated alerts to Slack and/or Teams to keep track of critical events and changes within DataHub.
- Hacktoberfest Winners Announced - We’ll recap this year’s Hacktoberfest and announce three winners of a $250 Amazon gift card & DataHub Swag.
Oct 2022
Agenda
- Conquer Data Governance with DataHub’s Metadata Tests - Learn how to tackle Data Governance with incremental, automation-driven governance using Metadata Tests provided in DataHub’s DataHub Cloud offering
- Community Case Study - The Grab Team shares how they are using DataHub for data discoverability, automated classification and governance workflows, data quality observability, and beyond!
- Upcoming Ingestion Sources - We’ll tell you the ins and outs of our upcoming dbt Cloud and Unity Catalog connectors
- Sneak Peek! Saved Views - Learn how you can soon use Saved Views to help end-users navigate entities in DataHub with more precision and focus
- Performance Improvements - Hear about the latest upgrades to DataHub performance
Sep 2022
Agenda
- Column Level Lineage is here! - Demo of column-level lineage and impact analysis in the DataHub UI
- Community Case Study - The Stripe Team shares how they leverage DataHub to power observability within their Airflow-based ecosystem
- Sneak Peek! Automated PII Classification - Preview upcoming functionality to automatically identify data fields that likely contain sensitive data
- Ingestion Improvements Galore - Improved performance and functionality for dbt, Looker, Tableau, and Presto ingestion sources
Aug 2022
Agenda
- Community Case Study - The Etsy Team shares their journey of adopting DataHub
- Looker & DataHub Improvements - surface the most relevant Looks and Dashboards
- Home Page Improvements to tailor the Browse experience
- Unified Ingestion Summaries - View live logs for UI-based ingestion and see historical ingestion reports across CLI and UI-based ingestion
- Patch Support - Native support for PATCH in the metadata protocol to support efficient updates to add & remove owners, lineage, tags and more
- Sneak Peek! Advanced Search
Jul 2022
Agenda
- Community Updates
- Project Updates
- Improvements to UI-Based Ingestion
- Sneak Preview - Bulk Edits via the UI
- Streamlined Metadata Ingestion
- DataHub 201: Metadata Enrichment
Jun 2022
Agenda
- Community Updates
- Project Updates
- dbt Integration Updates
- CSV Ingestion Support
- DataHub 201 - Glossary Term Deep Dive
May 2022
Agenda
- Community Case Study: Hear how the G-Research team is using Cassandra as DataHub’s Backend
- Creating & Editing Glossary Terms from the DataHub UI
- DataHub User Onboarding via the UI
- DataHub 201: Impact Analysis
- Sneak Peek: Data Reliability with DataHub
- Metadata Day Hackathon Winners
Apr 2022
Agenda
- Community Case Study: Hear from Included Health about how they are embedding external tools into the DataHub UI
- New! Actions Framework: run custom code when changes happen within DataHub
- UI Refresh for ML Entities
- Improved deletion support for time-series aspects, tags, terms, & more
- OpenAPI Improvements
Mar 2022
Agenda
- Community Case Study: Hear from Zendesk about how they are applying “shift left” principles by authoring metadata in their Protobuf schemas
- RBAC Functionality: View-Based Policies
- Schema Version History - surfacing the history of schema changes in DataHub's UI
- Improvements to Airflow Ingestion, including Run History
- Container/Domain-Level Property Inheritance
- Delete API
Feb 2022
Agenda
- Lineage Impact Analysis - using DataHub to understand the impact of changes on downstream dependencies
- Displaying Data Quality Checks in the UI
- Roadmap update: Schema Version History & Column-Level Lineage
- Community Case Study: Managing Data Lineage via YAML
Jan 2022
Agenda
- Community & Roadmap Updates by Maggie Hays (DataHub)
- Project Updates by Shirshanka Das (DataHub)
- Community Case Study: Adding Dataset Transformers by Eric Cooklin (Stash)
- Demo: Data Domains & Containers by John Joyce (DataHub)
- DataHub Basics — Data Profiling & Usage Stats 101 by Maggie Hays & Tamás Németh (DataHub)
- Demo: Spark Lineage by Mugdha Hardikar (GS Lab) & Shirshanka Das
Dec 2021
Agenda
- Community & Roadmap Updates by Maggie Hays (DataHub)
- Project Updates by Shirshanka Das (DataHub)
- 2021 DataHub Community in Review by Maggie Hays
- DataHub Basics -- Users, Groups, & Authentication 101 by Pedro Silva (DataHub)
- Sneak Peek: UI-Based Ingestion by John Joyce (DataHub)
- Case Study — DataHub at Grofers by Shubham Gupta
- Top DataHub Contributors of 2021 - Maggie Hays
- Final Surprise! We Interviewed a 10yo and a 70yo about DataHub
Nov 2021
Agenda
- Community & Roadmap Updates by Maggie Hays (DataHub)
- Project Updates by Shirshanka Das (DataHub)
- DataHub Basics -- Lineage 101 by John Joyce & Surya Lanka (DataHub)
- Introducing No-Code UI by Gabe Lyons & Shirshanka Das (DataHub)
- DataHub API Authentication by John Joyce (DataHub)
- Case Study: LinkedIn pilot to extend the OSS UI by Aikepaer Abuduweili & Joshua Shinavier
Oct 2021
Agenda
- DataHub Community & Roadmap Update - Maggie Hays (DataHub)
- October Project Updates - Shirshanka Das (DataHub)
- Introducing Recommendations - John Joyce & Dexter Lee (DataHub)
- Case Study: DataHub @ hipages - Chris Coulson (hipages)
- Data Profiling Improvements - Surya Lanka & Harshal Sheth (DataHub)
- Lineage Improvements & BigQuery Dataset Lineage by Gabe Lyons & Varun Bharill (DataHub)
Sep 2021
Agenda
- Project Updates and Callouts by Shirshanka
- GraphQL Public API Annoucement
- Demo: Faceted Search by Gabe Lyons (DataHub)
- Stateful Ingestion by Shirshanka Das & Surya Lanka (DataHub)
- Case-Study: DataHub @ Adevinta by Martinez de Apellaniz
- Recent Improvements to the Looker Connector by Shirshanka Das & Maggie Hays (DataHub)
- Offline
- Foreign Key and Related Term Mapping by Gabe Lyons (Acryl Data) video
Aug 2021
Agenda
- Project Updates and Callouts by Shirshanka
- Business Glossary Demo
- 0.8.12 Upcoming Release Highlights
- Users and Groups Management (Okta, Azure AD)
- Demo: Fine Grained Access Control by John Joyce (DataHub)
- Community Case-Study: DataHub @ Warung Pintar and Redash integration by Taufiq Ibrahim (Bizzy Group)
- New User Experience by John Joyce (DataHub)
- Offline
- Performance Monitoring by Dexter Lee (DataHub) video
Jul 2021
Agenda
- Project Updates by Shirshanka
- Release highlights
- Deep Dive: Data Observability: Phase 1 by Harshal Sheth, Dexter Lee (DataHub)
- Case Study: Building User Feedback into DataHub by Melinda Cardenas (NY Times)
- Demo: AWS SageMaker integration for Models and Features by Kevin Hu (DataHub)
Jun 2021
Agenda
- Project Updates by Shirshanka
- Release notes
- RBAC update
- Roadmap for H2 2021
- Demo: Table Popularity powered by Query Activity by Harshal Sheth (DataHub)
- Case Study: Business Glossary in production at Saxo Bank by Sheetal Pratik (Saxo Bank), Madhu Podila (ThoughtWorks)
- Developer Session: Simplified Deployment for DataHub by John Joyce, Gabe Lyons (DataHub)
May 2021
Agenda
- Project Updates by Shirshanka - 10 mins
- 0.8.0 Release
- AWS Recipe by Dexter Lee (DataHub)
- Demo: Product Analytics design sprint (Maggie Hays (SpotHero), Dexter Lee (DataHub)) - 10 mins
- Use-Case: DataHub on GCP by Sharath Chandra (Confluent) - 10 mins
- Deep Dive: No Code Metadata Engine by John Joyce (DataHub) - 20 mins
- General Q&A and closing remarks
Apr 2021
Agenda
- Welcome - 5 mins
- Project Updates by Shirshanka - 10 mins
- 0.7.1 Release and callouts (dbt by Gary Lucas)
- Product Analytics design sprint announcement (Maggie Hayes)
- Use-Case: DataHub at DefinedCrowd (video) by Pedro Silva - 15 mins
- Deep Dive + Demo: Data Lineage! Airflow, Superset integration (video) by Harshal Sheth and Gabe Lyons - 10 mins
- Use-Case: DataHub Hackathon at Depop (video) by John Cragg - 10 mins
- Observability Feedback share out - 5 mins
- General Q&A and closing remarks - 5 mins
Mar 2021
Agenda
- Welcome - 5 mins
- Project Updates (slides) by Shirshanka - 10 mins
- 0.7.0 Release
- Project Roadmap
- Demo Time: Themes and Tags in the React App! by Gabe Lyons - 10 mins
- Use-Case: DataHub at Wolt (slides) by Fredrik and Matti - 15 mins
- Poll Time: Observability Mocks! (slides) - 5 mins
- General Q&A from sign up sheet, slack, and participants - 10 mins
- Closing remarks - 5 mins
Feb 2021
Medium Digest Agenda
- Welcome - 5 mins
- Latest React App Demo! (video) by John Joyce and Gabe Lyons - 5 mins
- Use-Case: DataHub at Geotab (video) by John Yoon - 15 mins
- Tech Deep Dive: Tour of new pull-based Python Ingestion scripts (slides,video) by Harshal Sheth - 15 mins
- General Q&A from sign up sheet, slack, and participants - 15 mins
- Closing remarks - 5 mins
Jan 2021
Agenda
- Announcements - 2 mins
- Community Updates (video) - 10 mins
- Use-Case: DataHub at Viasat (slides,video) by Anna Kepler - 15 mins- Tech Deep Dive: GraphQL + React RFCs readout and discussion (slides ,video) by John Joyce and Arun Vasudevan - 15 mins
- General Q&A from sign up sheet, slack, and participants - 15 mins
- Closing remarks - 3 mins
- General Q&A from sign up sheet, slack, and participants - 15 mins
- Closing remarks - 5 minutes
Dec 2020
Agenda
- Quick intro - 5 mins
- Why did Grofers choose DataHub for their data catalog? by Shubham Gupta - 15 minutes
- DataHub UI development - Part 2 by Charlie Tran (LinkedIn) - 20 minutes
- General Q&A from sign up sheet, slack, and participants - 15 mins
- Closing remarks - 5 minutes
Nov 2020
Agenda
- Quick intro - 5 mins
- Lightning talk on Metadata use-cases at LinkedIn by Shirshanka Das (LinkedIn) - 5 mins
- Strongly Consistent Secondary Index (SCSI) in GMA, an upcoming feature by Jyoti Wadhwani (LinkedIn) - 15 minutes
- DataHub UI overview by Ignacio Bona (LinkedIn) - 20 minutes
- General Q&A from sign up sheet, slack, and participants - 10 mins
- Closing remarks - 5 minutes
Sep 2020
Agenda
- Quick intro - 5 mins
- Data Discoverability at SpotHero by Maggie Hays (SpotHero) - 20 mins
- Designing the next generation of metadata events for scale by Chris Lee (LinkedIn) - 15 mins
- General Q&A from sign up sheet, slack, and participants - 15 mins
- Closing remarks - 5 mins
Aug 2020
Agenda
- Quick intro - 5 mins
- Data Governance look for a Digital Bank by Sheetal Pratik (Saxo Bank) - 20 mins
- Column level lineage for datasets demo by Nagarjuna Kanamarlapudi (LinkedIn) - 15 mins
- General Q&A from sign up sheet and participants - 15 mins
- Closing remarks - 5 mins
Jul 2020
Agenda
- Quick intro - 5 mins
- Showcasing new entities onboarded to internal LinkedIn DataHub (Data Concepts, Schemas) by Nagarjuna Kanamarlapudi (LinkedIn) - 15 mins
- Showcasing new Lineage UI in internal LinkedIn DataHub By Ignacio Bona (LinkedIn) - 10 mins
- New RFC Process by John Plaisted (LinkedIn) - 2 mins
- Answering questions from the signup sheet - 13 mins
- Questions from the participants - 10 mins
- Closing remarks - 5 mins
June 2020
Agenda
- Quick intro - 5 mins
- Onboarding Data Process entity by Liangjun Jiang (Expedia) - 15 mins
- How to onboard a new relationship to metadata graph by Kerem Sahin (Linkedin) - 15 mins
- Answering questions from the signup sheet - 15 mins
- Questions from the participants - 10 mins
- Closing remarks - 5 mins
May 2020
Agenda
- Quick intro - 5 mins
- How to add a new aspect/feature for an existing entity in UI by Charlie Tran (LinkedIn) - 10 mins
- How to search over a new field by Jyoti Wadhwani (LinkedIn) - 10 mins
- Answering questions from the signup sheet - 15 mins
- Questions from the participants - 10 mins
- Closing remarks - 5 mins
Apr 2020 (17th)
Agenda
- Quick intro - 5 mins
- DataHub Journey with Expedia Group by Arun Vasudevan (Expedia) - 10 mins
- Deploying DataHub using Nix by Larry Luo (Shanghai HuaRui Bank) - 10 mins
- Answering questions from the signup sheet - 15 mins
- Questions from the participants - 10 mins
- Closing remarks - 5 mins
Apr 2020 (3rd)
- Agenda
- Quick intro - 5 mins
- Creating Helm charts for deploying DataHub on Kubernetes by Bharat Akkinepalli (ThoughtWorks) - 10 mins
- How to onboard a new metadata aspect by Mars Lan (LinkedIn) - 10 mins
- Answering questions from the signup sheet - 15 mins
- Questions from the participants - 10 mins
- Closing remarks - 5 mins
Mar 2020 (20th)
Agenda
- Quick intro - 5 mins
- Internal DataHub demo - 10 mins
- What's coming up next for DataHub (what roadmap items we are working on) - 10 mins
- Answering questions from the signup sheet - 15 mins
- Questions from the participants - 10 mins
- Closing remarks - 5 mins