Hey guys! Have a kubernetes run question. I have a use case where I need to dynamically set a kubernetes run config. I want to find out what the latest tag is of a docker image, and then use that same docker image for each task in a flow. I don’t know what the latest tag is at registration time, only at run time. Using :latest does not work, because it is not impossible :latest could change between the first task in a flow and the last task in a flow. Is there a recipe for doing this?

Hi @Zach Schumacher, does the docker image live in a container registry or on your local?

it is in docker hub

My best idea is to scrape the tag from the dockerhub page. I don’t think their API supports this operation.

no thats not my problem. The problem is i need to dynamically register the run config of the flow. I can get the latest tag from github no problem.

I know what you mean. I’ll look into this.

the answer can be “you need to figure that out with k8s and its not a prefect problem,” but just wanted to see if anyone else had that requirement before

I am a bit confused why something like the code snippet here doesn’t work? get_tag is a function here, not a task.

def get_tag():
    # do something
with Flow("test") as flow:
    # do something

flow.run()
flow.run_config = KubernetesRun(image=get_tag())

doesn’t that run when a flow gets registered

Tasks don’t get run but Python functions will.

right, so i want the get_tag code to be ran at flow run time

I see. That is not possible but the best advice I can give you then is using the Docker Sidecar and managing the containers this way.

i think this is exactly what i want, thanks!

Sorry for the detours 😅. Hope you get this working.