As promised in Anna's release Prefect 2.6.4 release announcement, I just published a post introducing the new Azure Container Instances infrastructure block and showing how to set it up to run your flows: https://medium.com/the-prefect-blog/serverless-prefect-flows-with-azure-container-instances-f2442ebc9343

This looks really neat! Time for us to start thinking about whether we actually need Kubernetes...

🤫

Does this create a ACI resource per deployment? Or can they use the same ACI resource? @Ryan Peden

Hi Thomas, it creates a container group with a single container per flow run. This makes it easy to change infrastructure parameters like CPU, memory, and GPU without needing to redeploy anything. If you want something that stays running on ACI and runs flows as they come in, you might prefer running an agent on ACI and setting up your flows to run as processes in the agent's container: https://discourse.prefect.io/t/how-to-run-a-prefect-2-agent-in-an-azure-container-instance/1427 If there are other ACI usage patterns you think make sense for Prefect flows, we'd love to hear more about them.

Thank you from the reply @Ryan Peden. From my understanding a container group can't spread across more than one host node, so won't this potentially put a upper limit on how many flows that can run in parallel with this infra? https://learn.microsoft.com/en-us/azure/container-instances/container-instances-region-availability#linux-container-groups Another pattern could be to have a 1:1 mapping between a Deployment in prefect and a ACI resource. Then you know that there is at least that amount of resources for the flow.

Hi Thomas, it's a container group + container per individual flow run, not per flow, so having multiple flows (or even multiple runs of one flow) running in parallel should only be limited by the quotas listed on this page - and with a paid subscription, you can request increases to those.

Oh I thought it was only possible to have one container group per ACI resources. Thank you for clarifying.

Hi @Ryan Peden. Thank you for great article. Unfortunately I am struggling second day to set it up: the healthcheck flow run stays in "late" state forever e.g. no container was created. Also I can not see any logs anywhere. Do you have some tip of how to debug it?

Hi, Ivan! When I've seen that happen it's often because something goes wrong in the container before the flow run can start. A couple of questions: • Are you using the default image, or a custom image? • Does anything show up in your agent logs?

I tried both with my own on ACR and with default one. Currently staying on the default. Logs are empty

Are you able to access the machine where the agent runs and see the agent's terminal output? It should be writing messages to the terminal when it picks up the flow run and tries to start an ACI container group to run the flow.

Probably I missed something (it is my second week doing with Prefect). I assumed that deployments would be run on Container Instances triggered directly from Prefect Cloud. So one still needs an Agent/VM, that listens to a queue and fires az cli commands under the hood to spin up containers? upd. Now it makes sense to me. Thanks)

That's right, although we use the Azure Python SDK instead of running CLI commands. It's part of Prefect's hybrid model. Prefect Cloud orchestrates your flow runs, but all the execution happens on VMs or hardware you control. I wrote a short tutorial on how to run an agent on ACI if you want to run everything there: https://discourse.prefect.io/t/how-to-run-a-prefect-2-agent-in-an-azure-container-instance/1427

Thank you, Ryan.

Hopefully last question to this thread: I managed to set up and running an agent on ACI. The plan is, that this Agent starts flows in new ACI containers. I am using custom image for agent and the same image I'm goint to use for "worker" containers. Azure container registry is deployed in the same resource group as container instances. I created Service Principal and assigned "Contributor" role on resource group to him. This SP is also stored as ACI Credentials block in Prefect. However, when I starting the flow, Agent Container throws the following error:

Message: The image '...' in container group '...' is not accessible. Please check the image and registry credential.

Have I missed something?

A couple of options come to mind: • Create a Docker Registry block with your ACR details, then selecting that registry from the drop down box in your ACI block's 'Image Registry` property. • Use the ACI block's 'Identities' field to add a user-assigned managed identity to the container groups the ACI block runs. ◦ Note: I've tested this feature on several types of Azure resources, but not ACR. However according to the docs, it should work.

Option 1 works. So Docker Registry Block may be used to store ACR credentials as well. I can’t thank you enough.

Happy to assist, Ivan. If you have any other questions, feel free to post in #prefect-azure to ensure maximum visibility 😄