Robin Weiß
10/21/2022, 9:33 AMRunning
state and start a K8s job & pod, the other 100 Flow Runs just idle around in “Pending”. I have searched everywhere to find hints about why they won’t start. There is definitely enough K8s computing resources for them to be scheduled, there is no concurrency limits set via tags or on the work queue directly. The K8s work queue just shows them as “Pending” in the UI. Does anyone have any idea where else to look? Cheers 🙂redsquare
10/21/2022, 9:36 AMRobin Weiß
10/21/2022, 9:37 AMredsquare
10/21/2022, 9:37 AMThis works, but only around 50 of my pods actually get into a running state, the other 100 just idle around in “Pending”
Robin Weiß
10/21/2022, 9:40 AMredsquare
10/21/2022, 9:40 AMRobin Weiß
10/21/2022, 9:43 AMredsquare
10/21/2022, 9:46 AMRobin Weiß
10/21/2022, 9:47 AMredsquare
10/21/2022, 10:03 AMRobin Weiß
10/21/2022, 10:07 AMredsquare
10/21/2022, 10:20 AMRobin Weiß
10/21/2022, 10:52 AMredsquare
10/21/2022, 11:02 AMRobin Weiß
10/21/2022, 12:37 PMreplicas: 1 _# We're using SQLite, so we should only run 1 pod_
Christopher Boyd
10/21/2022, 1:10 PMRobin Weiß
10/24/2022, 7:37 AM["prefect", "agent", "start", "-q", "kubernetes"]
command? Am I mixing stuff up here? Cheers!Christopher Boyd
10/25/2022, 12:34 PMRobin Weiß
10/25/2022, 1:38 PM13:37:34.302 | DEBUG | prefect.agent - Checking for flow runs...
13:37:34.524 | INFO | prefect.agent - Submitting flow run '<SAME_FLOW_RUN_ID>'
This keeps re-appearing 100 timesChristopher Boyd
10/25/2022, 2:12 PMRobin Weiß
10/25/2022, 2:42 PMChristopher Boyd
10/25/2022, 4:24 PMRobin Weiß
10/26/2022, 1:03 PM