for i in some_list:
try:
do_a_thing(i)
except SomeException as ex:
handle_exception(i, ex)
raise
I want to refactor "do_a_thing" to be a task and put
do_a_thing.submit(i)
in the body of the loop to let a couple threads do the work concurrently.
Is the exception handler enough for the parent task to wait for the results of the subtasks or do I need to explicitly wait for it? The subtask has no return value to check.
c
Chris White
10/02/2024, 7:43 PM
if it were me I'd move that exception handling logic into an
on_failure
state handler for the
do_a_thing
task.
To directly answer your question though - you do need to explicitly wait for the results of tasks that are submitted to task runners, either using
.wait()
method on the future itself or the
wait(...)
utility that allows you to wait for a group of futures.
j
Janet Carson
10/02/2024, 7:44 PM
Agree on_failure is the best way - but I'm still trying to make the code run with or without prefect until the prefect-side of things has a little more functionality.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.