Olivér Atanaszov
01/03/2023, 5:47 PMFlow(..., on_failure=cleanup_fn)
does not trigger the function for cancellation. Oh and I'm using Prefect 1.0.Bianca Hoch
01/03/2023, 11:21 PMOlivér Atanaszov
01/04/2023, 9:49 AMKubernetesRun
on different nodes:
def cleanup(flow, state):
# cleanup resources in K8s
# kubectl delete jobs --all --namespace $USER
with Flow("myflow", ..., on_failure=cleanup) a flow:
# do some work, e.g. run various child flow runs on K8s
create_flow_run("kubernetes_job")
create_flow_run("kubernetes_job")
# ...
Now, if I cancel the "main" flow named "myflow" in this example, I would like to run the cleanup
function to make sure all resources are released.
Does this make sense?Bianca Hoch
01/09/2023, 9:56 PM@resource_manager
decorator? It allows for custom setup and cleanup of cloud resources like kubernetes deployments (more docs on that here, if needed). There is an example of how to use that in the article posted above as well.on_failure
flow parameter will be called when a flow enters a failed state, and not a Cancelled
state. You could try creating your resource_manager
task, passing it to a state handler function which checks to see if the the flow is_finished
(finished captures the Cancelled
state), and then passing the state handler to the flow.Olivér Atanaszov
01/10/2023, 3:29 PM