yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-09-30 11:26:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add threat model and incident response (#23603)

Browse Source
This commit is contained in:
Sriharsha Chintalapani 2025-09-28 13:17:23 -07:00 committed by GitHub
parent ab11983741
commit ed144ab0e8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 626 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

305
INCIDENT_RESPONSE.md Normal file
View File

@ -0,0 +1,305 @@
# OpenMetadata Incident Response Plan
This document outlines the incident response procedures for security issues in the OpenMetadata project.
## Scope
This incident response plan covers:
- All code within the OpenMetadata organization repositories
- Security vulnerabilities in OpenMetadata services
- Metadata exposure incidents
- Supply chain security issues
- Infrastructure compromises
## Incident Lead
**Primary Incident Lead**: @harshach
- Responsible for coordinating incident response
- Decision authority for security releases
- External communication coordination
**Backup Incident Leads**: @pmbrull, @mohityadav766, @tutte
## Reporting Security Issues
### How to Report
All security issues must be reported privately through one of these channels:
1. **GitHub Security Advisories** (Preferred)
- Navigate to: https://github.com/open-metadata/OpenMetadata/security/advisories
- Click "Report a vulnerability"
- Provide detailed information
2. **Email**: security@open-metadata.org
- Encrypt sensitive details using our PGP key (available on our website)
### What to Include
- Detailed description of the vulnerability
- Steps to reproduce
- Potential impact assessment
- Affected versions
- Any proof-of-concept code (if applicable)
### What NOT to Do
- **DO NOT** create public GitHub issues for security vulnerabilities
- **DO NOT** disclose vulnerabilities on public forums or social media
- **DO NOT** submit pull requests with security fixes without prior coordination
## Response Timeline
- **Initial Response**: Within 24 hours
- **Impact Assessment**: Within 72 hours
- **Resolution Target**: Within 30 days for critical issues
- **Public Disclosure**: Coordinated after fix is available
## Incident Response Phases
### Phase 1: Detection & Initial Response (0-24 hours)
1. **Acknowledge Receipt**
- Send confirmation to reporter
- Assign tracking identifier
- Engage incident lead (@harshach)
2. **Initial Assessment**
- Verify the vulnerability
- Determine severity using CVSS scoring
- Identify affected versions
- Check if actively exploited
3. **Immediate Containment** (if critical)
- Disable affected features if possible
- Notify cloud service providers if applicable
- Revoke compromised credentials
### Phase 2: Investigation & Coordination (24-72 hours)
1. **Form Response Team**
- Incident Lead: @harshach
- Security Engineer(s)
- Affected component maintainer(s)
- Communications coordinator
2. **Deep Investigation**
- Root cause analysis
- Full impact assessment
- Check for similar vulnerabilities
- Review logs for exploitation attempts
3. **Develop Fix**
- Create patches for supported versions
- Develop test cases
- Security review of proposed fix
### Phase 3: Mitigation & Remediation
1. **Prepare Releases**
- Patch all supported versions
- Update documentation
- Prepare security advisory
2. **Coordinate Disclosure**
- Notify major users under embargo (if applicable)
- Request CVE assignment
- Coordinate disclosure date with reporter
3. **Deploy Fixes**
- Release patched versions
- Update Docker images
- Update Helm charts
- Deploy to cloud services
### Phase 4: Communication
1. **Internal Communication**
- Update internal teams
- Brief support teams
- Update runbooks
2. **External Communication**
- Publish security advisory
- Update security page
- Send notifications to mailing list
- Social media announcements (if critical)
3. **Credit & Acknowledgment**
- Credit reporter (unless anonymity requested)
- Update security acknowledgments page
### Phase 5: Post-Incident Review
1. **Incident Review** (Within 1 week)
- Timeline review
- Response effectiveness
- Lessons learned
- Process improvements
2. **Security Improvements**
- Update security practices
- Enhance testing procedures
- Update threat model
- Security training needs
3. **Documentation Updates**
- Update incident response plan
- Update security documentation
- Share learnings with community
## Severity Levels
### Critical (CVSS 9.0-10.0)
- Complete system compromise
- Unauthorized access to all metadata
- Remote code execution
- Authentication bypass
- **Response Time**: Immediate, fix within 24-48 hours
### High (CVSS 7.0-8.9)
- Significant metadata exposure
- Privilege escalation
- Connection credential exposure
- **Response Time**: Within 7 days
### Medium (CVSS 4.0-6.9)
- Limited metadata exposure
- Denial of service
- Information disclosure
- **Response Time**: Within 30 days
### Low (CVSS 0.1-3.9)
- Minor information leakage
- Difficult to exploit issues
- **Response Time**: Next regular release
## Special Considerations for OpenMetadata
### Metadata-Specific Incidents
Since OpenMetadata handles only metadata, not actual data:
1. **Metadata Exposure**
- Assess what metadata was exposed
- Determine business impact (not data breach)
- Notify affected teams about architecture exposure
2. **Connection Configuration**
- Immediately rotate any exposed credentials
- Audit connected system access logs
- Verify read-only permissions maintained
3. **Lineage Information**
- Assess business process exposure
- Review competitive sensitivity
- Update access controls
### Supply Chain Incidents
1. **Dependency Vulnerabilities**
- Immediate assessment of exposure
- Patch or workaround deployment
- Update dependency management
2. **Connector Compromises**
- Disable affected connectors
- Audit ingestion logs
- Verify metadata integrity
## Contact Information
### Security Team
- **Email**: security@open-metadata.org
- **GitHub Security**: https://github.com/open-metadata/OpenMetadata/security
- **Incident Lead**: @harshach
### Escalation Path
1. Security Team
2. Incident Lead (@harshach)
3. OpenMetadata Maintainers
4. Collate (parent organization) if required
## Communication Templates
### Initial Response
```
Subject: Security Report Acknowledged - [TRACKING-ID]
Thank you for reporting this security issue. We take all security reports seriously.
We have assigned tracking ID [TRACKING-ID] to your report and will investigate immediately.
Expected timeline:
- Initial assessment: Within 72 hours
- Resolution target: Within 30 days
We will keep you updated on our progress. Please keep this issue confidential until we coordinate disclosure.
Thank you for helping keep OpenMetadata secure.
```
### Security Advisory Template
```
# Security Advisory: [CVE-ID]
**Affected Component**: OpenMetadata [Component]
**Severity**: [Critical/High/Medium/Low]
**CVSS Score**: [Score]
**Affected Versions**: [Versions]
**Fixed Versions**: [Versions]
## Summary
[Brief description]
## Impact
[Detailed impact assessment]
## Mitigation
[Steps to mitigate if patch cannot be immediately applied]
## Fix
[How to upgrade/patch]
## Credit
Discovered by [Reporter Name/Organization]
## References
- [CVE Link]
- [Additional References]
```
## Security Disclosure Policy
- We request 90 days to address reported vulnerabilities
- We coordinate disclosure timing with reporters
- We credit reporters unless anonymity is requested
- We maintain a security acknowledgments page
- We follow responsible disclosure practices
## Training & Preparedness
### Regular Activities
- Quarterly incident response drills
- Annual security training for maintainers
- Regular dependency audits
- Penetration testing (annually)
### Required Training
- OWASP Top 10 awareness
- Secure coding practices
- Incident response procedures
- Communication protocols
## References
- [GitHub Security Advisories](https://docs.github.com/en/code-security/security-advisories)
- [CVE Process](https://cve.mitre.org/)
- [FIRST PSIRT Framework](https://www.first.org/standards/frameworks/psirts/)
- [NIST Incident Response Guide](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-61r2.pdf)
---
*Last Updated: 2025-01-28*
*Version: 1.0*
*Next Review: Quarterly*

266
THREAT_MODEL.md Normal file
View File

@ -0,0 +1,266 @@
# OpenMetadata Threat Model
This document outlines the security threat model for OpenMetadata, a unified metadata platform for data discovery, observability, and governance.
## Important Scope Notice
**OpenMetadata is a metadata-only platform**. It does not:
- Store, process, or transmit actual data
- Execute queries against production data sources
- Modify data in connected systems
- Provide direct data access capabilities
OpenMetadata exclusively manages metadata - information *about* data, such as table schemas, column descriptions, data lineage, ownership, and quality metrics. This metadata-only architecture significantly reduces the attack surface and potential impact of security incidents.
## Overview
While OpenMetadata does not handle actual data, it only manages metadata that provides insights into an organization's data architecture, which requires appropriate security controls. This threat model identifies potential risks specific to a metadata management platform.
## Asset Inventory
### Primary Assets (Metadata Only)
1. **Business Metadata**
- Table and column descriptions
- Business glossary terms
- Data ownership information
- Team structures and responsibilities
- Tags and classifications
2. **Technical Metadata**
- Database and table schemas (structure only)
- Column names and data types
- Data lineage relationships
- Data quality metrics and test results
- Service topology information
3. **Operational Metadata**
- Ingestion pipeline configurations
- Connection configurations (credentials for metadata extraction only)
- User activity logs
- Search queries and access patterns
4. **Platform Components**
- OpenMetadata Server (REST APIs)
- Web UI (React application)
- Ingestion Framework (metadata collectors)
- Backend Database (MySQL/PostgreSQL)
- Search Index (Elasticsearch/OpenSearch)
## Threat Actors
### External Threats
- **Unauthorized Users**: Attempting to discover information about data architecture
- **Competitors**: Seeking insights into data organization and business processes
- **Reconnaissance Actors**: Gathering information for potential future attacks on actual data systems
### Internal Threats
- **Curious Employees**: Accessing metadata beyond their authorized scope
- **Departing Employees**: Exporting organizational knowledge
- **Compromised Accounts**: Valid accounts used inappropriately
## Threat Categories
### 1. Information Disclosure Threats
**T1.1: Metadata Reconnaissance**
- Risk: Exposure of data architecture and organization structure
- Impact: Provides blueprint for potential attacks on actual data systems
- Attack Vectors:
- Unauthorized API access
- Search interface exploitation
- Excessive permissions
**T1.2: Connection Configuration Exposure**
- Risk: Leakage of service endpoints and authentication methods
- Impact: Reveals connection patterns but not data access
- Attack Vectors:
- Configuration file access
- API endpoint enumeration
- Log file exposure
**T1.3: Business Process Disclosure**
- Risk: Exposure of data flows and business logic through lineage
- Impact: Competitive disadvantage or process manipulation
- Attack Vectors:
- Lineage graph traversal
- Glossary term extraction
- Ownership mapping
### 2. Authentication & Authorization Threats
**T2.1: Unauthorized Metadata Access**
- Risk: Users accessing metadata outside their domain
- Impact: Information leakage across business units
- Attack Vectors:
- Privilege escalation
- Role misconfiguration
- Token manipulation
**T2.2: Authentication Bypass**
- Risk: Anonymous access to metadata
- Impact: Full metadata exposure
- Attack Vectors:
- JWT vulnerabilities
- SSO misconfiguration
- Default credentials
### 3. Data Integrity Threats
**T3.1: Metadata Tampering**
- Risk: Modification of metadata leading to confusion or misdirection
- Impact: Incorrect business decisions based on false metadata
- Attack Vectors:
- API manipulation
- Direct database access
- Ingestion pipeline compromise
**T3.2: Lineage Manipulation**
- Risk: False data lineage information
- Impact: Compliance violations or incorrect impact analysis
- Attack Vectors:
- Ingestion tampering
- API exploitation
### 4. Availability Threats
**T4.1: Service Disruption**
- Risk: OpenMetadata platform unavailability
- Impact: Inability to discover or govern data assets
- Attack Vectors:
- DoS attacks
- Resource exhaustion
- Database overload
**T4.2: Search Index Corruption**
- Risk: Search functionality failure
- Impact: Degraded data discovery capabilities
- Attack Vectors:
- Index poisoning
- Bulk operation abuse
### 5. Supply Chain Threats
**T5.1: Dependency Vulnerabilities**
- Risk: Vulnerabilities in third-party libraries
- Impact: Platform compromise
- Attack Vectors:
- Known CVEs in dependencies
- Dependency confusion attacks
**T5.2: Connector Compromise**
- Risk: Malicious metadata ingestion connectors
- Impact: False metadata or reconnaissance
- Attack Vectors:
- Unofficial connector installation
- Connector impersonation
## Mitigations
### M1: Access Control
- Make sure OpenMetadata hosted as your organization's internal tooling. You don't need to expose to public internet. Lock it behind your company's VPN
- Implement Role-Based Access Control (RBAC) for metadata domains
- Use team-based metadata visibility
- Regular access review and certification
- Principle of least privilege for metadata access
- Audit logging of all metadata access
### M2: Authentication Security
- Enforce SSO/SAML for enterprise deployments
- Multi-factor authentication for administrative access
- Regular token rotation
- Session timeout policies
- Strong password requirements
### M3: Connection Security
- Encrypt stored connection configurations
- Use secret management systems for credentials
- Implement credential rotation
- Limit ingestion permissions to read-only metadata operations
- Network isolation for ingestion pipelines
### M4: Platform Hardening
- Regular security updates
- Dependency vulnerability scanning
- Container security scanning
- Secure default configurations
- Disable unnecessary features
### M5: Monitoring & Auditing
- Comprehensive audit logs for metadata changes
- Anomaly detection for unusual access patterns
- Real-time alerting for security events
- Regular security assessments
- Compliance reporting
### M6: Data Protection
- TLS for all network communications
- Encryption at rest for sensitive configurations
- Secure backup procedures
- Data retention policies
- Secure deletion practices
## Residual Risks
Even with mitigations, these risks remain:
1. **Zero-day vulnerabilities** in platform components
2. **Social engineering** targeting metadata administrators
3. **Advanced persistent threats** conducting long-term reconnaissance
4. **Insider threats** with legitimate access
5. **Supply chain compromises** in dependencies
## Security Best Practices
### For Deployment
1. Deploy in isolated network segments
2. Use read-only service accounts for metadata ingestion
3. Implement network policies restricting egress
4. Regular security patching schedule
5. Monitoring and alerting setup
### For Configuration
1. Disable default accounts
2. Configure appropriate session timeouts
3. Enable audit logging
4. Set up automated backups
5. Implement rate limiting
### For Operations
1. Regular access reviews
2. Security awareness training
3. Incident response procedures
4. Change management processes
5. Documentation of security configurations
## Compliance Considerations
While OpenMetadata doesn't store actual data, consider:
- **Metadata Privacy**: Some metadata might be considered sensitive
- **Access Logging**: Required for compliance audits
- **Change Tracking**: Metadata modification history
- **Data Governance**: Using OpenMetadata to support compliance programs
## Incident Response
For security incidents involving OpenMetadata:
1. Contain: Isolate affected systems
2. Assess: Determine scope of metadata exposure
3. Notify: Inform stakeholders of potential information disclosure
4. Remediate: Patch vulnerabilities and rotate credentials
5. Review: Update security controls based on lessons learned
## Conclusion
OpenMetadata's metadata-only architecture inherently limits security risks compared to data platforms. The primary concern is unauthorized information disclosure about data architecture rather than data breach. By implementing appropriate access controls and monitoring, organizations can safely leverage OpenMetadata for data discovery and governance while maintaining security.
## References
- [OpenMetadata Security Documentation](https://docs.open-metadata.org/deployment/security)
- [OWASP Top 10](https://owasp.org/www-project-top-ten/)
- [CIS Controls](https://www.cisecurity.org/controls)
---
*Last Updated: 2025-01-28*
*Version: 1.0*

21
bootstrap/sql/migrations/native/1.10.0/mysql/schemaChanges.sql
View File

@ -43,4 +43,23 @@ CREATE TABLE IF NOT EXISTS notification_template_entity (
UNIQUE KEY fqnHash (fqnHash),
INDEX idx_notification_template_name (name),
INDEX idx_notification_template_provider (provider)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
-- Optimize table listing queries by indexing the schema hash prefix
ALTER TABLE table_entity
ADD COLUMN databaseSchemaHash VARCHAR(768) CHARACTER SET ascii COLLATE ascii_bin
GENERATED ALWAYS AS (SUBSTRING_INDEX(fqnHash, '.', 3)) STORED;
CREATE INDEX idx_table_entity_schema_listing
ON table_entity (deleted, databaseSchemaHash, name, id);
-- Optimize stored procedure listing queries by indexing the schema hash prefix
ALTER TABLE stored_procedure_entity
ADD COLUMN databaseSchemaHash VARCHAR(768) CHARACTER SET ascii COLLATE ascii_bin
GENERATED ALWAYS AS (SUBSTRING_INDEX(fqnHash, '.', 3)) STORED;
ALTER TABLE stored_procedure_entity
DROP INDEX idx_stored_procedure_entity_deleted_name_id;
CREATE INDEX idx_stored_procedure_schema_listing
ON stored_procedure_entity (deleted, databaseSchemaHash, name, id);

24
bootstrap/sql/migrations/native/1.10.0/postgres/schemaChanges.sql
View File

@ -46,4 +46,26 @@ CREATE TABLE IF NOT EXISTS notification_template_entity (
);
CREATE INDEX IF NOT EXISTS idx_notification_template_name ON notification_template_entity(name);
CREATE INDEX IF NOT EXISTS idx_notification_template_provider ON notification_template_entity(provider);
CREATE INDEX IF NOT EXISTS idx_notification_template_provider ON notification_template_entity(provider);
-- Optimize table listing queries by indexing the schema hash prefix
ALTER TABLE table_entity
ADD COLUMN databaseSchemaHash VARCHAR(768)
GENERATED ALWAYS AS (
concat_ws('.', split_part(fqnhash, '.', 1), split_part(fqnhash, '.', 2), split_part(fqnhash, '.', 3))
) STORED;
CREATE INDEX idx_table_entity_schema_listing
ON table_entity (deleted, databaseSchemaHash, name, id);
-- Optimize stored procedure listing queries by indexing the schema hash prefix
ALTER TABLE stored_procedure_entity
ADD COLUMN databaseSchemaHash VARCHAR(768)
GENERATED ALWAYS AS (
concat_ws('.', split_part(fqnhash, '.', 1), split_part(fqnhash, '.', 2), split_part(fqnhash, '.', 3))
) STORED;
DROP INDEX IF EXISTS idx_stored_procedure_entity_deleted_name_id;
CREATE INDEX idx_stored_procedure_schema_listing
ON stored_procedure_entity (deleted, databaseSchemaHash, name, id);

15
openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/ListFilter.java
View File

@ -176,9 +176,18 @@ public class ListFilter extends Filter<ListFilter> {
public String getDatabaseSchemaCondition(String tableName) {
String databaseSchema = queryParams.get("databaseSchema");
return databaseSchema == null
? ""
: getFqnPrefixCondition(tableName, databaseSchema, "databaseSchema");
if (databaseSchema == null) {
return "";
}
if (!nullOrEmpty(tableName)
&& (tableName.equals("table_entity") || tableName.equals("stored_procedure_entity"))) {
String databaseSchemaHash = FullyQualifiedName.buildHash(databaseSchema);
queryParams.put("databaseSchemaHashExact", databaseSchemaHash);
return String.format("%s.databaseSchemaHash = :databaseSchemaHashExact", tableName);
}
return getFqnPrefixCondition(tableName, databaseSchema, "databaseSchema");
}
public String getServiceCondition(String tableName) {