Chris Martin07/30/2020, 3:24 PM
Laura Lorenz (she/her)07/30/2020, 3:42 PM
Chris Martin07/30/2020, 4:09 PM
@task def task_with_cleanup(x): try: // do work here except KeyboardInterrupt: // flow is being cancelled do cleanup here sys.exit()
Laura Lorenz (she/her)07/30/2020, 4:13 PM
Chris Martin07/30/2020, 4:16 PM
Laura Lorenz08/10/2020, 4:45 PM
Jim Crist-Harif08/10/2020, 5:07 PM
on cancellation is an implementation detail, and isn't something I would recommend relying on. If you need a resource to be cleaned up when a python process exits, I recommend looking into the
handlers in the Python standard library. We hope to update the UI in the next week to add a button to cancel a flow run, but in the meantime you can manually call the
graphql route to cancel an active flow run.
Chris Martin08/10/2020, 5:52 PM
mainly because I really want to detect that a cancellation has been requested and both
are somewhat abstracted away from that. Do you think there's any chance of adding an
. That way it would be much more explicit about what was going on.
Jim Crist-Harif08/10/2020, 5:59 PM
tasks in a resource manager always run if the
task runs, even if cancellation occurs. Definitely on our roadmap.
Is there any way I can get notified when that change is released (it'll save me scouring the ui for a cancellation button for the next week!)Definitely, I'll make a note to ping you when that's released.
Chris Martin08/10/2020, 6:54 PM
Definitely, I'll make a note to ping you when that's released.Awesome, much appreciated
Is there a reason you need to know that cancellation occurred specifically? What code do you want to run on cancellation that you wouldn't want to run in other cases?You may have a fair point here. I guess the common case here is controlling something like AWS batch with Prefect. You make an API call to submit a batch run and then make subsequent API calls to poll the status of the run so as to check for some terminal state. All of the terminal states guarantee that the resources acquired by the batch run have been released, so if the task runs to terminal then there's no cleanup to do. Incidentally, I don't think your current resource manager works great here because you don't know the resource that needs tearing down until the task's run() method has been called, although possibly I'm missing a pattern here! Now, what this means if the run() method has executed then there's no cleanup to do. If however, the run() method is interrupted (either by a cancellation or by some unhandelled exception) then we really want to stop the AWS batch job. I guess my argument in this case is that I'd much rather have an explicit callback than hook into a systemwide hook for a system I don't control. Moreover I think there is some value between being able to differentiate between a task cancellation (which I think we can assume means that the system is generally in a good state) and an an arbitrary termination (in which case you can't really know what state the system will be in).
Jim Crist-Harif08/19/2020, 10:43 PM
Definitely, I'll make a note to ping you when that's released.Hi @Chris Martin, the cancellation button is now available in the cloud UI (I think this went live late last week). Apologies for the late notice here.