Hey everyone, we are using Prefect with k8s. Our f...
# ask-community
j
Hey everyone, we are using Prefect with k8s. Our flow runs are always marked as "Failed" at the beginning and only later as "Running" and "Completed". I guess pods with flow runs don't start fast enough so Prefect marks them as "Failed". Is it possible to somehow increase this time limit? We get unnecessary notifications about failed flow runs. 😕
1
d
I read a comment on this somewhere. But you could set appropriate time diff in one of the following settings
Copy code
PREFECT_AGENT_PREFETCH_SECONDS='15' (from defaults)
PREFECT_WORKER_PREFETCH_SECONDS='10.0' (from defaults)
Not entirely sure where I read it though.
alternatively while running agent / worker from cli
--prefetch-seconds
is available.
v
Thanks for the tip @Deceivious. It seems our flow runs were transitioning from scheduled > failed > running because prefects k8s service account did not have rights to access
kube-system
namespace to determine cluster UID. Setting it explicitly via ENV as in here resolved it. The helm charts do not include the necessary rights AFAIK. 🙏
👍 1