I have a small question regarding dedicated execution of prefect tasks using a DaskExecutor. For example let's say I have some special tasks in a flow that need to be executed specifically on a dedicated machine. Is it possible to provide this information in Prefect?
03/30/2022, 10:33 AM
Could you explain the problem you try to solve? Otherwise, I'll give you answers that will only confuse you further.
03/30/2022, 1:55 PM
Yes I present to you Prefect’s least used feature!! Look at the DaskExecutor docs, you can use worker resources annotation to point a task to a specific worker. For example, you have a worker with a GPU
04/01/2022, 8:28 AM
Hahah, Thank you! This solved the problem. I was experimenting with a much more difficult workaround of creating flows with a different executors on the dedicated machines for these tasks and then creating an orchestrating flow, but the Worker Resources are much simpler. Thank you!!🙏