https://prefect.io logo
p

Peter Peter

10/28/2021, 2:46 PM
Hey, Is there any documentation on how prefect and Kubernetes work together? It is a little unclear if all flows run get submitted to Kubernetes even if there is no capacity or if prefect queues these and submits the job when capacity exists, If it is Perfect that manages this is it Kubernetes agent that deals with this? TIA
k

Kevin Kho

10/28/2021, 2:52 PM
Hey @Peter Peter, Prefect starts the flow as Kubernetes jobs. If it can’t get the hardware needed, the flow will likely not start. You will see Lazarus resubmit the flow for execution. This commonly happens to users using Spot Instances for their Kubernetes cluster when they aren’t able to get those instances. You can read more about Lazarus here
@Marvin open “Lazarus Docs say there are 10 retries but this should be 3”
p

Peter Peter

10/29/2021, 6:38 PM
@Kevin Kho Are the number of retries configurable? and also is the time between retries configurable?
k

Kevin Kho

10/29/2021, 6:41 PM
You can change this on server. Time between doesnt seem exposed
I think it’s here
Also might be tied to the heartbeat cutoff here
2 Views