Hi folks I am starting to consider using Prefect in a projec

Question

Hi folks I am starting to consider using Prefect in a project but one thing remains to be clarified is that currently possible to specify the worker ressource requirement for a task running on a Dask cluster E g I have a flow with two tasks A and B I would like to specify that A should run only on GPU workers while B should run on NO GPU workers I did not find anything about that in the docs

Oli Steadman · Answer

Wow I would be keen to know this too

Chris Hart · Answer

if I understand correctly isn t Dask only used for the cloud offering

Jeremiah · Answer

Good news Dask is available everywhere For a tutorial see <https medium com the prefect blog prefect dask 2da6cdb23ee2>

Jeremiah · Answer

< Romain> it s an excellent question We ve largely focused on affinity at the flow level by using the `DaskExecutor` to specify a specific execution cluster We have considered using task tags to allow affinity at the task level as well basically reusing the task tags as Dask resources However it s something we d like to make sure we support for users whether they use Dask or not and we re still iterating on the best way to enable that

Romain · Answer

< Jeremiah> Thanks for the answer So if I understand correctly there is no clean way to do it for now

Oli Steadman · Answer

Is there an issue we can track in Github etc for notifications on this feature

Romain · Answer

I guess I could also simply build my own flow runner that would pass as keywords arguments the ressource requirements from the tasks According to this <https github com PrefectHQ prefect blob master src prefect engine flow runner py L418> it would be done by passing ```ressources=task ressources``` to the submit method from the executor I would simply need to add the ressources at the Task definition level

Chris White · Answer

Hey everyone in this thread < Oli Steadman> < Romain> gt this is now possible see the docs here

Romain · Answer

< Chris White> Wonderful Thank you