mirror of
https://github.com/datahub-project/datahub.git
synced 2025-06-27 05:03:31 +00:00
docs(iceberg): added notes on session expiry config, some minor updates (#13832)
This commit is contained in:
parent
a8c5202993
commit
1964bcf4d1
@ -353,6 +353,19 @@ When setting up your AWS role, you'll need to configure a trust policy. Here's a
|
||||
}
|
||||
```
|
||||
|
||||
### Vended Credentials and Session Expiry
|
||||
|
||||
When creating a warehouse, the session expiry duration for the vended credentials can be configured via the
|
||||
`datahub iceberg` cli by passing the `--duration_seconds` parameter. If not specified, this currently defaults to 3600
|
||||
seconds. This needs to be a minimum of 900 seconds (15 minutes).
|
||||
|
||||
With AWS, at the time of creating the role for use with the catalog, a `MaxSessionDuration` can also be set. AWS currently
|
||||
supports values between 1 hour to 12 hours. This configuration will set the upper limit for what can be configured for the
|
||||
warehouse via `--duration_seconds` flag.
|
||||
|
||||
See `datahub iceberg create --help` for the flags and default if not specified. This can also be updated using
|
||||
`datahub iceberg update`
|
||||
|
||||
### Namespace and Configuration Requirements
|
||||
|
||||
1. **Namespace Creation**: A namespace must be created before any tables can be created within it.
|
||||
@ -522,18 +535,15 @@ A: Check that:
|
||||
## Known Limitations
|
||||
|
||||
- AWS S3 only support
|
||||
- Concurrency - supports single instance of GMS
|
||||
- Multi-table transactional `/commit` is not yet supported
|
||||
- Table operation purge is not yet implemented
|
||||
- Metric collection and credential refresh mechanisms are still in development
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Improved concurrency with multiple replicas of GMS
|
||||
- Support for Iceberg multi-table transactional `/commit`
|
||||
- Additional table APIs (scan, drop table purge)
|
||||
- Azure and GCP Support
|
||||
- Namespace permissions
|
||||
- Enhanced metrics
|
||||
- Credential refresh
|
||||
- Proxy to another REST Catalog to use its capabilities while DataHub has real-time metadata updates.
|
||||
|
Loading…
x
Reference in New Issue
Block a user