How do you handle model registry / deployment for Prefect deployments? TL;DR we want revisions and be able to rollout / rollback a ML model for a daily running deployment. We have a Kedro data science inference pipeline that runs for 3 weeks (RRule). During the second week we would like to update the reference to the training model (the path to the pickle / mlflow run id) that is associated with the Prefect deployment. Is there best practice to do that? Kedro is using MLFlow for experiment manager, but I’m not sure how to propagate the pickle / mlflow run id to the deployed Prefect deployment. I could reinvent the wheel and add an S3 pointer to the “latest” pickle, but I want Amazon SageMaker / MLFlow like capability to review and deploy models.

deleting my previous answer because I realize I misread your question

I asked ChatGPT this prompt

In MLFlow, how do you rollout / rollback different revisions of a model?

Obviously I could sneak behind Prefect’s back and call:

mlflow.update_registered_model()

But I am concerned of chaos and inability to review, track and manage the different model deployments associated with a Prefect daily inference pipeline.

I'm not super familiar with mlflow but you have a couple options for updating parameters that get used inside of a prefect flow

Thanks @Jake Kaplan! Does Prefect version control it / show a history of parameters changed?

You can only see the active parameters for a deployment (not the change history) But you can see each individual flow run from that deployment should have the parameters that got used

Thanks Jake, appreciate the answer. This is an MLOps / ML engineering question to be more precise.

forgive a potentially naive question, since MLOps isn't my domain of expertise, but I'm curious what specific things you'd be looking for Prefect to offer that you cannot find in the version control offered by GitHub (assuming that your model versions are defined in code) i see this

where you can track changes made to deployments (to the states / deployments of the Kubernetes cluster).

and I feel like you could create artifacts that represent infra events or model execution that you could track over time, but I'm wondering if I missing part of your motivation on this

I’m not sure how to propagate the pickle / mlflow run id to the deployed Prefect deployment.

could you put the reference in the

description

of the deployment ? then itd be markdown rendered in the UI on the deployment's page

Thanks @Nate. I’m wondering how Uber for example rolls out and rolls back data science models in production.

@Ofir One idea is using variables. You can store the model version in a variable and update it to the new one, or update it to the previous one and roll it back. This variable could also dictate coded logic if it changes across models.