prefect flow enters Crashed state as soon as it displays downloading flow code and then importing flow code .the output logs only shows exited with status code -11. This happens on deployment on a queue which is long running continuously in background however when we start a new agent and queue then the same flow runs fine Slack Conversation

Hi, what type of infrastructure block is being used for the flow that's failing?

i created a dummy infra block with default settings

what is the block type name?

process-test

How far into development with Prefect are you? If things are still flexible, I'd recommend going with workers over agents, that way you don't have to create storage and infra blocks for your deployments, and the creating deployments is a bit simpler as a result.

i am creating work pools and my queues lie inside the work pool its the 'default-agent-pool' within which i have my queue running

Right, and you have an agent connected to that pool?

yes which is running and keeping the queue in healthy state

Instead of

default-agent-pool

, you could create a typed work pool and start a worker with a matching type. Workers differ from agents in that they manage infrastructure of a specific type for their flow runs. With the way your agent is set up right now, it's a bit difficult to pin down what the issue might be.

new agent on a new queue does not cause the issue

Is there any problem with switching your deployment over to that new queue then?

for the production running queue thay i have i do not need to start my agent everytime and can schedule my runs on it but when i create ad hoc queues i need to start the agent everytime

Do you have multiple deployments on your production queue? If so are they all encountering this issue, or just one?

on already running agent and its queues i just need to deploy my flow and background running agent already picks it up

Where is your agent running? Is it in a cloud service or on your own hardware? What operating system?

on our own hardware with linux servers

Did your flow runs work successfully in the past, or have you been unable to get them working at all?

the flow runs successfully only on ad hoc queues created but never on the production running queue

Are you creating the ad hoc queues/agents on the same linux server as production?

yes it is on same server

🤔 I thought it might be something related to the environment being different where the production agent is running, since issues with process blocks/process workers are usually indicative of things like the flow code not being available, or the python env not being set up correctly

so what i usually do where i am failing is call the deployment first 1) "prefect deployment build -n <name> name -q queuename entryscriptname.py -ib process/process-test --apply 2) apply deployment on runnung queue 3) create a quick run through UI hiwever if i run the deloyment first step 1) 2) start the already running agent again by command - prefect agent start -p 'default-agent-pool' or prefect agent start -q 'queue name' 3) create a quick run through UI then the process runs fine

I'm still not clear on what the difference between those two is. How was the production agent started?

venv/bin/prefect agent start --work-queue "workqueuue name"