yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-29 09:42:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/ingestion/tests/integration/mlflow
History
Pere Miquel Brull a7c8056bf5
[issue-1079] - MlFlow Connector (#1677) 
* Prepare infra

* Store experiment information in MySQL & MinIO

* Use CreateMlModelEntityRequest instead of MlModel for PUT operations

* Update MlFlow infra

* Prepare MlFlow source

* Prepare Mlflow workflow

* Simplify test and prepare README

* Revert compose

* Fix compose

* Prepare warnings and fix features

* Use non-default port for integration test

* Use mlflow-skinny for the client application
2021-12-11 18:40:19 -08:00
..
__init__.py
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
docker-compose.yml
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
Dockerfile
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
experiment.py
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
Makefile
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
README.md
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
train.py
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00

README.md

MlFlow Integration Test

We have prepared a small test to check the MlFlow ingestion.

We have used a decoupled architecture for MlFlow with:

  • mlflow running in a remote server
  • minio as the artifact store
  • mysql as the registry

To run this test:

  • cd into this directory
  • make build
  • pip install mlflow-skinny sklearn. We use the skinny one for the client.
  • python experiment.py should show new experiments in http://localhost:5000
  • python train.py will register a new model
  • metadata ingest -c examples/workflows/mlflow.json will run the workflow.