2020-02-11 14:14:57 -08:00
|
|
|
# DataHub Roadmap
|
2020-02-11 12:25:33 -08:00
|
|
|
|
2020-02-11 14:14:57 -08:00
|
|
|
Below is DataHub's roadmap for the short and medium term. We'll revise this on a regular basis and welcome suggestions from the communities.
|
2020-02-11 12:25:33 -08:00
|
|
|
|
2020-02-11 13:36:38 -08:00
|
|
|
## Short term (3-6 months)
|
2020-02-15 06:26:37 -08:00
|
|
|
### Replace PDSC with PDL
|
|
|
|
|
- Simplified syntax + annotations
|
|
|
|
|
### Entity-specific MCE & MAE
|
|
|
|
|
- Spilt up unified event for scalability & modularity
|
2020-02-15 06:48:53 -08:00
|
|
|
### Schemas as an entity
|
2020-02-14 13:26:53 -08:00
|
|
|
- Make schemas searchable
|
|
|
|
|
- Support GraphQL shcmeas
|
2020-02-11 12:25:33 -08:00
|
|
|
### Compliance management for datasets
|
2020-02-15 06:26:37 -08:00
|
|
|
- Simple tag-based compliance metadata
|
2020-02-15 06:48:53 -08:00
|
|
|
### Gremlin-based Query DAO
|
|
|
|
|
- Support majority of gremlin-compatible graph DBs
|
2020-02-11 12:25:33 -08:00
|
|
|
### Kubernetes migration
|
2020-02-15 06:48:53 -08:00
|
|
|
- Migration from docker-compose to [Kubernetes](https://kubernetes.io/) for Docker container orchestration
|
2020-02-11 12:25:33 -08:00
|
|
|
### Azure deployment
|
2020-02-15 06:48:53 -08:00
|
|
|
- Run DataHub in [Azure](https://azure.microsoft.com/en-us/) and provide a how-to guide for Azure deployment
|
2020-02-11 12:25:33 -08:00
|
|
|
|
2020-02-15 06:26:37 -08:00
|
|
|
## Medium term (6 months - 1 year)
|
2020-02-11 12:25:33 -08:00
|
|
|
### Fine grain lineage
|
2020-02-15 06:26:37 -08:00
|
|
|
### Operational metadata
|
|
|
|
|
- Indexing in OLAP store (Pinot) with TTL
|
2020-02-11 12:25:33 -08:00
|
|
|
### Social features
|
2020-02-15 06:48:53 -08:00
|
|
|
- Users will be able to like and follow entities
|
|
|
|
|
### Micro-services as an entity
|
|
|
|
|
### Kundera-based Local DAO
|
|
|
|
|
- Support a wide range of document stores
|
2020-02-11 12:25:33 -08:00
|
|
|
### Integration tests
|
2020-02-15 06:48:53 -08:00
|
|
|
- Add docker-based integration tests.
|
2020-02-11 12:25:33 -08:00
|
|
|
### AWS deployment
|
2020-02-15 06:48:53 -08:00
|
|
|
- Run DataHub in [AWS](https://aws.amazon.com/) and provide a how-to guide for AWS deployment
|
2020-02-15 06:26:37 -08:00
|
|
|
|
|
|
|
|
## Long term (1+ year)
|
|
|
|
|
### Rewrite midtier in Node
|
2020-02-15 06:48:53 -08:00
|
|
|
|
