Chris Martin
07/30/2020, 3:24 PMLaura Lorenz (she/her)
07/30/2020, 3:42 PMChris Martin
07/30/2020, 4:09 PM@task
def task_with_cleanup(x):
try:
// do work here
except KeyboardInterrupt:
// flow is being cancelled do cleanup here
sys.exit()
Laura Lorenz (she/her)
07/30/2020, 4:13 PMChris Martin
07/30/2020, 4:16 PMLaura Lorenz
08/10/2020, 4:45 PMJim Crist-Harif
08/10/2020, 5:07 PMKeyboardInterrupt
on cancellation is an implementation detail, and isn't something I would recommend relying on. If you need a resource to be cleaned up when a python process exits, I recommend looking into the atexit
or weakref.finalize
handlers in the Python standard library. We hope to update the UI in the next week to add a button to cancel a flow run, but in the meantime you can manually call the cancel_flow_run
graphql route to cancel an active flow run.Chris Martin
08/10/2020, 5:52 PMkeyboardInterrupt
mainly because I really want to detect that a cancellation has been requested and both atexit
or weakref.finalize
are somewhat abstracted away from that. Do you think there's any chance of adding an on_cancel
method to Task
. That way it would be much more explicit about what was going on.Jim Crist-Harif
08/10/2020, 5:59 PMcleanup
tasks in a resource manager always run if the setup
task runs, even if cancellation occurs. Definitely on our roadmap.Is there any way I can get notified when that change is released (it'll save me scouring the ui for a cancellation button for the next week!)Definitely, I'll make a note to ping you when that's released.
Chris Martin
08/10/2020, 6:54 PMDefinitely, I'll make a note to ping you when that's released.Awesome, much appreciated
Is there a reason you need to know that cancellation occurred specifically? What code do you want to run on cancellation that you wouldn't want to run in other cases?You may have a fair point here. I guess the common case here is controlling something like AWS batch with Prefect. You make an API call to submit a batch run and then make subsequent API calls to poll the status of the run so as to check for some terminal state. All of the terminal states guarantee that the resources acquired by the batch run have been released, so if the task runs to terminal then there's no cleanup to do. Incidentally, I don't think your current resource manager works great here because you don't know the resource that needs tearing down until the task's run() method has been called, although possibly I'm missing a pattern here! Now, what this means if the run() method has executed then there's no cleanup to do. If however, the run() method is interrupted (either by a cancellation or by some unhandelled exception) then we really want to stop the AWS batch job. I guess my argument in this case is that I'd much rather have an explicit callback than hook into a systemwide hook for a system I don't control. Moreover I think there is some value between being able to differentiate between a task cancellation (which I think we can assume means that the system is generally in a good state) and an an arbitrary termination (in which case you can't really know what state the system will be in).
Jim Crist-Harif
08/19/2020, 10:43 PMDefinitely, I'll make a note to ping you when that's released.Hi @Chris Martin, the cancellation button is now available in the cloud UI (I think this went live late last week). Apologies for the late notice here.