Hey all! I recently came across this page (https://orion-docs.prefect.io/concepts/task-runners/) that showed a way of using dask.annotation to set the priority of each task, but I can't get it to work.
Since I couldn't import DaskTaskRunner for the life of me I tried using LocalDaskExecutor, but I'm not sure if it does the same thing:
Copy code
@task
def file_write(x):
print(x)
with open('test_result.txt', 'a') as f:
f.write(str(x))
return x
with Flow("priority_test", executor=LocalDaskExecutor()) as flow:
with dask.annotate(priority=-10):
written1 = file_write(2)
with dask.annotate(priority=0):
written2 = file_write(1)
with dask.annotate(priority=10):
written3 = file_write(0)
I expected it to write "012", but am consistently getting "201"
k
Kevin Kho
04/04/2022, 8:55 PM
Are you using Prefect 1 or 2? DaskTaskRunner is for Prefect 2 and DaskExecutor is for Prefect 1
z
Zanie
04/04/2022, 9:45 PM
The Dask priority annotations will only work with Prefect v2
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.