Looking to get back into prefect now that it is more stable. I have multiple deployments each with their own flow. Can i prevent the same flow from running twice at the same time while using just one work queue? In the early betas the solution was one work queue or agent per deployment. Edit: dang. Just noticed #5623 is still open

does setting the Flow Run Concurrency on a work queue to 1 not work for this?

I want multiple flows to run at the same time, but not the same flow twice. (Overlap is possible if the flow is taking too long or someone tries to run it manually)

now that queues are part of the deployment for a flow (as of yesterdays 2.1 release) I feel like this should be possible

Ok yeah, i saw something in the changelogs about how work queues were changed. You are probably right

Can i prevent the same flow from running twice at the same time while using just one work queue?

that's what concurrency limits are for, but yeah 100% with the new work queue setup you can solve that by assigning a new work queue name per deployment, solving it in a really simple way

Alright I must be missing something. The new deployment yaml allows me to specify the work queue name, which is great, but wouldn’t it be better if we could also specify the concurrency options? Or what would be the best way to set that new work queues concurrency limit to 1

manually in the UI (lame) or via the REST API I guess

That’s got to be a pain to detect. I think for my situation i will use the work queue name option, but also have a little shell script to create the work queues ahead of time. Also I mentioned adding work queue options to the deployment, but the more i think about it the more fishy that sounds design wise.

wouldn’t it be better if we could also specify the concurrency options

you can do so if you create a work queue yourself using one more command:

prefect work-queue create x --limit 1

this is expected - you need to be a bit more careful here, we don’t have ML yet predicting typos

it should throw an error instead

we decided against that - we create a queue if it doesn't exist this follows our principle of incremental adoption

Haha it’s alright clint

if that's helpful, you can first list available queues and only create deployment if your queue exists

prefect work-queue ls

Jared, I'm not sure if it works for your use case, but Prefect 2.1 added Python deployments, so you could do something like:

import asyncio
from flows import my_flow
from prefect.deployments import Deployment
from prefect.client import get_client

deployment_name = "my_flow"
work_queue_name = "my_flow_work_queue"

async def run_deployment():
    # create and apply the deployment
    deployment = await Deployment.build_from_flow(
        flow=test_flow,
        name=deployment_name,
        work_queue_name=work_queue_name,
        version=1
    )
    await deployment.apply()

    # set the work queue concurrency
    client = get_client()
    work_queue = await client.read_work_queue_by_name(work_queue_name)
    await client.update_work_queue(work_queue.id, concurrency_limit=1)

asyncio.run(run_deployment())

CLI deployments are usually simpler, but I wanted to mention this option in case it is useful!

Ah great! Missed the get by name function.