I have a flow function decorated with @flow and specify a DaskTaskRunner. I thought this would cause my tasks to get run on my Dask cluster but I saw this in your documentation:
βTo run your task with a task runner, you must call the task with .submit().β
Is this still true even if my flow function is configured as described above?
β 1
m
Mason Menges
09/14/2022, 9:13 PM
Hey @Tony Piazza that's correct they do need to be called using .submit to benefit from using the dask task runner
"To run tasks asynchronously use the
.submit
method when you call them. If you call a task as you would normally in Python code it will run synchronously, even if you are calling the task within a flow that uses the
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.