A question to the prefect community. Is there a w...
# ask-community
t
A question to the prefect community. Is there a way where certain to execute certain task runners on a task runner other than the one specified by the main flow? My use case is pretty simple. A lot of my tasks require many cores, but some long running tar + g jobs only need a single core. I would like to try to specific a separate dask task runner that only uses a single core per work, rather than the dask task runner using 32 per worker. I have previously looked into the insides of a
task
class before, and see that there is a mechanism to specify a task runner - this internal option is what denoted the blocking vs non-blocking behaviour when directly calling a task vs calling a tasks
.submit
method. For instance: https://github.com/PrefectHQ/prefect/blob/6c7233bbe55fbe554956bbc70fbfa31de12c1543/src/prefect/tasks.py#L550 seems to suggest I could extend either the
__call__
or
.submit
method to pass through a different task runner. Would this work?