https://prefect.io logo
#prefect-community
Title
# prefect-community
s

Scott McCallen

05/10/2022, 8:34 PM
Hello! I have a question regarding executing flows using the Kubernetes Agent. I have created a Job template for the flow run container following the example(s) provided in KubernetesRun documentation. My understanding is that the Job template is then encoded in the flow metadata registered with the Prefect server when I run the
prefect register
command. There are a couple of scenarios where I want to change aspects of the flow run container based on Flow Parameters. For instance, I'd like the flow container to have additional Job annotations in some runs of the flow, but not all runs of the flow. I'm not sure how to do this dynamically when the flow is started. The only thing I've been able to figure out so far is that I need multiple registrations of the same flow - each registration having a different Kubernetes Job template. Is there a dynamic way to adjust/modify the Job template for a given registered flow?
k

Kevin Kho

05/10/2022, 8:44 PM
You are right that these are held in the metadata on registration so you can’t really change it based on parameters. The only way to do this is to have a parent Flow that calls
create_flow_run
and
create_flow_run
actually takes a RunConfig so you can change the RunConfig on the fly
Thiis static registration is inherently limiting and why things are being made more dynamic in 2.0
s

Scott McCallen

05/10/2022, 8:48 PM
Okay, thanks for the confirmation. I've been looking at the KubernetesAgent implementation and haven't been able to come up with a way to make it work. Seems like I need to try a different approach.
k

Kevin Kho

05/10/2022, 8:49 PM
The way above will work I think because the parent flow will take parameters
s

Scott McCallen

05/10/2022, 8:51 PM
Okay, I'll take a look at that
Thanks!
5 Views