Ka Wing Fong
12/15/2023, 8:39 PMMarvin
12/15/2023, 8:39 PMKa Wing Fong
12/15/2023, 8:41 PMcancelling
state?Nate
12/15/2023, 8:42 PMwhat do I do when my flow get stuck at@Ka Wing Fong did the infrastructure get deleted?state?cancelling
Marvin
12/15/2023, 8:42 PMprefect flow-run cancel
, followed by the ID of the flow run:
bash
$ prefect flow-run cancel 'your-flow-run-id'
Alternatively, you can cancel a flow run from the UI by navigating to the flow run's detail page and clicking the 'Cancel' button in the upper right corner.
The cancellation process should move the flow run to a "Cancelling" state, and the agent monitors the state of flow runs and detects that cancellation has been requested. The agent then sends a signal to the flow run infrastructure, requesting termination of the run. If the run does not terminate after a grace period (default of 30 seconds), the infrastructure will be killed to ensure that the flow run exits. This process is designed to be robust and provides a strong guarantee of cancellation.
It's important to note that this cancellation process requires that an agent is running to enforce the cancellation. If you notice that the cancellation does not happen after these steps or if the flow run remains stuck, it is advisable to consult the Prefect community on Slack or raise a Github issue to get further assistance.
Let me know if you need more help with this issue, such as raising a Github issue or exploring other potential solutions.Ka Wing Fong
12/15/2023, 8:43 PMImagePullBackOff
Ka Wing Fong
12/15/2023, 8:43 PMNate
12/15/2023, 8:44 PMKa Wing Fong
12/15/2023, 8:44 PMFailed to pull image "us-docker.pkg.dev/shiftsmart-api/ssm-data-pipeline/ssm-data-pipeline:None": rpc error: code = NotFound desc = failed to pull and unpack
Seeing this error but we never set the tag to None
in our CI 😞Nate
12/15/2023, 8:44 PMdescribe
a pod thats stuck in ImagePullBackOff?Nate
12/15/2023, 8:45 PMimage
tag None
?Ka Wing Fong
12/15/2023, 8:47 PMKa Wing Fong
12/15/2023, 8:48 PMNate
12/15/2023, 8:48 PMKa Wing Fong
12/15/2023, 8:49 PMKa Wing Fong
12/15/2023, 8:49 PMNate
12/15/2023, 8:50 PMKa Wing Fong
12/15/2023, 8:50 PMKa Wing Fong
12/15/2023, 8:50 PMKa Wing Fong
12/15/2023, 8:59 PMNate
12/15/2023, 9:42 PM