Hi Everyone. I am running prefect in a Kubernetes cluster. I am using Dask as executor and for the run environment using Kubernetes Run. I am using the cloud and in the trial. What I found is, I have a lag between 4-65 sec between job scheduled and job running. Is there any way we can tune the scheduler or the worker for running with minimum lags? Let me know if I have to check some config for this.

Hey @Prabin Mehta, unfortunately the answer here with Kubernetes is that there is an expected latency triggered by many factors, but not specifically related to Prefect, and usually beyond our control.

And is the lag same when using DaskExecutor?

DaskExectutor will have a lag if it needs to spin up and install images on workers. If you’re connecting to an existing cluster, there might be less.

as kevin said, going to be releated to if youre using spot instances, what your image pull policy is, how large your image is, how long your container takes to start up, etc.