Hi, I have a question about Prefect agents and Flows : is there something in Prefect such as Kubernetes'

resources

parameters to specify resource affinities (RAM, CPU cores, GPU environment) a Flow requires to be run ? Ex (Kubernetes) :

resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

Thanks !

Hey @flavienbwk, not super sure if this is what you need but the RunConfig takes in resource requests like

flow.run_config = KubernetesRun(cpu_request=2, memory_request="2Gi")

as seen here . If you need affinity, for example you need a flow to run on a specific node, you can use nodeAffinity in the job template. Does that answer you?

Thank you for this option ! I would like to use exclusively Prefect and its agents instead of Kubernetes jobs. Does Prefect support auto-detection of its agents resources (CPU cores count, RAM count) and is able to support the same parameters as for

KubernetesRun

?

Auto-detection I think is not supported, but maybe you can try setting it high and see what happens? I guess the pod startup might fail if it can’t get the resources?

I mean I would like to avoid using Kubernetes. And exclusively use VM-runned Prefect agents.

Ah I see if you use the LocalAgent or DockerAgent, it just keeps starting flows without any knowledge of the resources or utilization.

Thank you for these precisions ! Might be an idea for future Prefect features 🙂

What would you want the behavior to be? Since the LocalAgent is already designed to use as much resources as possbiel?

My configuration is the following : I have 5 VPS with different capabilities (some with lot of RAM, some with low RAM, some high CPU cores, low CPU cores). Each VPS has 1 Prefect agent instanciated. I would like Prefect to automatically be able to distribute the flow I've registered with minimum requirements as parameter (

(cpu_request=2, memory_request="2Gi")

) to an agent with sufficient resources.

Gotcha, I think the prerequisite of that is that it’s possible to set limits with Python on a process that is started, which I’m not sure it is for all OSs. Will take a look a it.

Let me refine my thinking : • If Prefect API distributes tasks to agents : Prefect API should know which agent has which resources (RAM/CPU) to send the Flow to the agent with enough Flow's requested resources. • If agents listen to Prefect API tasks : Agent should make sure it is able to accept the Flow depending on its resources and Flow's requested resources.

I understand the intention but to be honest, this can be managed by a user with labels, so I don’t know when a feature like this would be picked up. Also, Cloud has the task run concurrency and flow run concurrency which queue flows/tasks if too many with the same label/tag are already running. This is the first time I’ve seen this request. I’ll keep an eye out if I see it pop up more.

Yes labels are really useful. For example with agents having GPU capabilities (label e.g:

cuda

). And this is what I'll be using for the moment while I don't have Kubernetes. BUT, for quantities-related characteristics (such as RAM amount or CPU count), it can't really be used.

Hi! Just to add some additional context, resource based scheduling is considered out of scope for Prefect. Scheduling jobs to consume resources optimally is a very complex problem that is better addressed by dedicated tooling (like Kubernetes).

Hi @Zanie thank you for these addings. My problem could be addressed just by implementing "countable tags" like cpu-xx or ram-xx or vram-xx. It might just be tags provided by the user (declarative values in agent's config). These countable tags would make it possible to perform comparison operations on them (greater than, equal, lower than...).

As Kevin has noted we do support this via concurrency limiting by tag.

Do you have any link in the documentation that points to this? Thanks

Yeah OK it's a Cloud feature. Thanks for the link