Hannah Amundson

07/01/2020, 7:32 PM
Hello! We are using KubernetesJobEnvironment. From what I can tell, this is the process: click run -> agent creates job -> job creates job with our kubernetes job environment Is this correct? Secondly, it seems to take quite a lot of time to create our kubernetes job environment job. Is this how it always is or does it have to do with our configurations? If it is always this way, is there a way to speed it up?


07/01/2020, 7:36 PM
Hi @Hannah Amundson! Yes you are correct in your thinking that the agent created an initial prefect job and then that job creates the prefect job. Generally slower start times are due to Kubernetes having to pull the image and partition the resources. How long of a time delay are you seeing? Also, I’m not sure of your set up but you may not need the KubernetesJobEnvironment and instead could use the LocalEnvironment which will run your flow in that initial prefect job using your configured executor. The Kubernetes agent has some configuration options which can be set on that prefect-job:
--mem-request               TEXT    Requested memory for Prefect init job
      --mem-limit                 TEXT    Limit memory for Prefect init job
      --cpu-request               TEXT    Requested CPU for Prefect init job
      --cpu-limit                 TEXT    Limit CPU for Prefect init job
      --image-pull-policy         TEXT    imagePullPolicy for Prefect init job
      --service-account-name      TEXT    Name of Service Account for Prefect init job

Hannah Amundson

07/01/2020, 8:30 PM
ok thank you so much!

Chris White

07/02/2020, 5:10 AM
@Marvin archive “How to speed up the KubernetesJobEnvironment”