https://prefect.io logo
Title
a

Aaron Goebel

08/18/2022, 12:10 AM
is there a way to use
Parameters
or something analogous as input to a
KubernetesRun
to request a specific amount of resources on the fly? I'm working w/ Fargate and want to give users flexibility to run flows with differing amounts of compute
👀 1
e

Emil Christensen

08/18/2022, 2:11 PM
Hi @Aaron Goebel 👋 Am I correct in assuming that you’re using Prefect 1.0? Could you tell me a bit more about your use case? When you create a flow run you should be able to override values of your run config, including memory and cpu: 1. Manually - in the UI, navigate to the flow you want to run, then go to the “🚀 Run” tab and then “Show advanced run configuration” 2. Programmatically - instantiate a client and use create_flow_run passing a KubernetesRun run_config with your desired cpu_request.
💯 1
a

Aaron Goebel

08/19/2022, 8:02 PM
I'm using Prefect 1 yes. The use case is that some pipelines, depending on input parameters, would require more resources to run. Also, potentially a gpu. This would change what the KubernetesRun provisions
I have the prefect agent in default namespace running on EKS backed with fargate (currently no GPU support)
for the scenario with a GPU, it's my understanding that I would need to provision an EC2 reserved node group in a separate kubernetes namespace
Also, Is it possible to use the default namespaced agent to dispatch jobs to the namespace that has GPU resources?
e.g. can I interleave flows that would run on fargate and ec2? My goal is to be able to stitch together flows, on the fly, underneath parent flows. And the execution instance (fargate, ec2) for individual subflows underneath the parent flow may be different
e

Emil Christensen

08/20/2022, 3:11 PM
I think it should be possible to selectively place flows on different clusters and/or capacity providers, but I’ll have to look into it more. Another approach would be to just run two agents, each pointing to a different k8s run config. Since the agents aren’t doing the work themselves, they require very few resources to run. Then you can use labels to direct work to the appropriate agent.
👀 1
🙌 1