We actually got our Prefect 2 flows working in our production environment yesterday! 🎉 However, today's big daily scheduled orchestrator job has a problem I haven't seen during my 2.5 weeks of development and testing: Two of the created flows do not get picked up by the agent, they're stuck in the work queue. I just tried to start another deployment and now there's 3 in the queue. All the previous flows were picked up correctly and they all have the same tags. Has anyone run into a similar problem? I'm not sure what other information should include.. The agent is still running. There are no concurrency limits set for this work queue.

Can you share how did you start your agent and how did you create your work queue? (I'm sure you did it right, just a sanity check)

Had to take a call and grab some tea, let's see now..

just wanted to leave 😂 will stay for a bit then

Haha, no worries! I'll try things out and report back, one of your suggestions will probably fix the issue.

I would try instead:

prefect agent start -t "transformation-queue-prod"

Oh, the queue isn't tagged with that string though, that's actually the name of the queue. The queue is tagged with "prod" and "transformation". Is -t still the way to go?

not to confuse you: • in 2.0.4 the tags were the matching criteria to match deployments with work queues - you would start an agent with:

prefect agent start -t prod

• in 2.0.5 which we'll release this week, we will do it via work queue names -- the tags will still work for backward compatibility but it will become less confusing - - you would start an agent with:

prefect agent start -q prod

- queue stands for queue and it will create a queue "prod" if it doesn't exist yet

Ah, but I need to use multiple tags.. Unless I redo my tag infrastructure a bit. 🙂

I'll try restarting the agent. It'd prefer not to, since the main orchestration flow is still "running" and waiting for these scheduled jobs to and finish, but it shouldn't be a problem.

you can start a new agent process without interrupting the one with active runs - you can have multiple agents polling from the same work queue

oh, true

no, you can keep all your tags - in 2.0.4 only one tag on your deployment e.g. "prod" is enough for runs from that deployment to be picked up from an agent started with

prefect agent start -t prod

Ah, right. I want this particular agent to only pick up things tagged with prod and transformation though, since we have other deployments tagged with prod and integration. Integrations are picked up by an agent on a different vm with more ram (for lifting more data), that has access to our on-prem databases. While the transformation vm/agent only runs dbt and similar commands.. If I start the agent with two tags, will it automatically listen to queues matching those two? Or is that not supported?

it will poll for both as you described:

Awesome, thank you. I'll see if it helps compared to using a queue name, just being slow because I'm having some ssh issues with the vm..

nice work! thanks for the update