Hi, I'm running into an issue that's been posted a few times before...
first there's a warning...
Copy code
[WARNING] Task run 'bcc56e8e-a1bd-4145-abdb-aad866d63c6f' received abort during orchestration: The enclosing flow must be running to begin task execution. Task run is in PENDING state.
then an error:
Copy code
[ERROR] prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
Previous advice has been that this is caused by an upstream problem, but I can't see anything in the logs (journalctl on ec2 instance) that would cause the enclosing flow to not be in a running state.
Our infra is an prefect agent running on ec2, spawning docker containers on the same box. I checked the monitoring on the box - no cpu/mem spikes.
I've added task retries, but I guess that's not going to fix it either, if the flow is already dead? What else should I look for?
z
Zanie
04/10/2023, 3:05 PM
You’ll probably want to look at the agent logs to see if the agent is reporting it as failed/crashed.
m
Mark NS
04/10/2023, 3:31 PM
I was looking in journalctl which should include logs from the agent no?
z
Zanie
04/10/2023, 3:56 PM
I’m not sure — seems like that would depend on your setup.
Zanie
04/10/2023, 3:56 PM
It should be fairly obvious which logs are from the agent.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.