Question for the prefect devs in the know.
Is there going to be upcoming support for allowing two separate task runners to support the processing of tasks within the one flow? I see that within the
Task
class the task runner attribute is close to being explosed.
I ask this because I have a set of tasks that require large compute resources, and others that require small. I would love to be able to define two different dask task runners to handle both of these use cases, and specify which to use when calling my task function. Something like
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.