I know for an ECS task, it is possible to specify ...
# prefect-community
a
I know for an ECS task, it is possible to specify the amount of RAM/CPU that is available to each task. Would it be possible to set these same constraints on individual tasks?
k
Like on the Prefect side right? Definitely not for now. Potentially in the future if we end up with a Ray Executor? Cuz Ray provides this functionality I think
Sorry the word task is overloaded here. I don’t know if you mean ECS task or Prefect task
a
I do apologize for the miscommunication. In this case it is a prefect task that will run on ECS. I run command line applications in the Nextflow engine that each have varying resource requirements. I was hoping there exited a way to 'limit' the resources needed for each individual prefect task such that a task that 'writes to a file' would not need the same resources that would require 100GB of RAM.
k
No it’s not your fault 😅. The only way to do this at the moment is splitting up the different steps into different Flows because you can specify resources on a Flow level (ECS task settings) and then you can chain them together with a “main flow”
a
That is an interesting workaround. I will look into that. Thank you for the pointer!