hi all, I’ve had a flow in prefect cloud that has been scheduled to start for ~1 hour. I know its not a label issue bc I see the k8s agent picked it up. Could very well be something in our k8s cluster but wanted to check if there was something else going on. Including the flow run ID in the thread.

Hey @Zach Schumacher, I just see Lazarus trying to start this. Was this flow running successfully before?

Is your k8s cluster made with preemptible vms?

I’m not sure what that means, lol

AWS spot instances?

OH a spot instanc

I’ve seen this behavior happen when the Flow is unable to run because it didn’t get compute resources.

it is not a spot instance atm, we are making that switch in the future tho

Maybe you can run the flow in debug mode so we can get more logs?

hm we aren’t using Dask so I don’t think its related. I’ll try setting debug on the job template to see if I get anything more meaningful

I think this as a way of inspecting what’s going on in k8s might be hekpful.

any change this fe831335-b4a8-45ce-8edc-6c6f4056e863 id gives anything on your side?

Not seeing anything on Prefect log side. I’m looking for any clue if Prefect can’t hit k8s.

I’m just a dumby lol. Its bc we have a custom entrypoint script and I wasn’t overriding it in my job template. DUH

Well glad you got it sorted out!