Hi, does anyone know how to get an agent to run a specific docker image from a docker repo where the docker image is defined in the deployment? Hence is the agent able to start up different docker images?

yes, if you're running a prefect agent in an env where docker is available, it will try to submit the flow run as a new container using the image specified by that deployment

by any chance do you have link to docs for that?

@Marvin can you find docs on the prefect agent and explain how it interacts with the DockerContainer infrastructure block associated with a given deployment

Ah, the mysteries of Prefect agents and DockerContainer infrastructure blocks shall be revealed. Prefect agents are responsible for deploying flow runs to the infrastructure specified by the deployment. They poll work pools for work and create execution environments for flow runs based on the infrastructure settings provided in the deployment source. DockerContainer infrastructure blocks are configurations that agents use to create Docker-based execution environments for flow runs. These blocks contain the settings needed to run flows in Docker containers source. When building a deployment, you can specify a custom infrastructure block using the

-ib

or

--infra-block

options, passing the type and name of the block in the format

type/name

, with

type

and

name

separated by a forward slash source. For example, when creating your deployment files, you can specify the DockerContainer infrastructure like this:

$ prefect deployment build ./my_flow.py:my_flow -n my-flow-deployment -t test -i docker-container -sb s3/my-bucket --override env.EXTRA_PIP_PACKAGES=s3fs

In this example, the DockerContainer infrastructure is specified in addition to a preconfigured AWS S3 bucket storage block source. So, in summary, Prefect agents interact with DockerContainer infrastructure blocks by using the settings provided in the block to create Docker-based execution environments for flow runs. I hope that clears up the fog of confusion. Good luck with your never-ending quest for knowledge, dear human.

this is mostly correct marvin hallucinated the t flag and the override syntax , more docs here

But this would mean we’d use the S3 storage to store the .py files and we have some base image to run the code in. I am looking more for something where the code and the docker image are already combined in the docker repo and you specify that.

Hmmm. Interesting. That would make sense. I have to look into that more I guess.

Thanks