https://prefect.io logo
Title
m

Mark NS

04/08/2023, 5:24 AM
Hi, I'm running into an issue that's been posted a few times before... first there's a warning...
[WARNING] Task run 'bcc56e8e-a1bd-4145-abdb-aad866d63c6f' received abort during orchestration: The enclosing flow must be running to begin task execution. Task run is in PENDING state.
then an error:
[ERROR] prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
Previous advice has been that this is caused by an upstream problem, but I can't see anything in the logs (journalctl on ec2 instance) that would cause the enclosing flow to not be in a running state. Our infra is an prefect agent running on ec2, spawning docker containers on the same box. I checked the monitoring on the box - no cpu/mem spikes. I've added task retries, but I guess that's not going to fix it either, if the flow is already dead? What else should I look for?
z

Zanie

04/10/2023, 3:05 PM
You’ll probably want to look at the agent logs to see if the agent is reporting it as failed/crashed.
m

Mark NS

04/10/2023, 3:31 PM
I was looking in journalctl which should include logs from the agent no?
z

Zanie

04/10/2023, 3:56 PM
I’m not sure — seems like that would depend on your setup.
It should be fairly obvious which logs are from the agent.