<@ULVA73B9P> Is it possible to create a automation...
# ask-marvin
b
@Marvin Is it possible to create a automation that checks whether there is any flow within a deployment that is running and if not start a run?
m
thought for 474.1 seconds
Short answer: Yes. In Prefect 3.x you can set up an Automation that “keeps at least one run alive” for a specific deployment. The pattern is: - Trigger: a proactive Metric trigger that evaluates “running flow runs for this deployment” and fires when the value drops below 1 for some period - Action: RunDeployment (start a new run of that deployment) How to set it up in the UI (recommended) 1) Go to Automations in the Prefect UI and create a new Automation. 2) Trigger - Type: Metric - Posture: Proactive - Metric: “Running flow runs” (filter by your specific deployment) - Condition: operator “<”, threshold “1” - Window: choose a small “within” window (e.g., 30–60s) and optionally “firing for” to avoid flapping 3) Action - Action: Run deployment - Select the same deployment - Optionally set parameters for the run 4) Optional: set a cooldown to prevent rapid re-firing if your infrastructure has a startup delay. Notes and alternatives - Automations are created via the UI or API; there is no “prefect automation create” CLI command. You can list/inspect/pause/resume via CLI though. See CLI: prefect automation. - If you just need runs at a fixed cadence (not conditional) a normal deployment schedule is simpler. - To ensure you don’t accidentally start multiple runs, pair this with a concurrency policy so you only ever run one at a time. See Global concurrency limits. - Under the hood, this relies on Prefect’s Event/Metric-based automations. Concept docs: Automations and triggers. If you prefer the API route, the Automation would include: - A Metric trigger that filters by your
deployment_id
and checks “running flow runs < 1” within a time window - An action of type RunDeployment targeting that
deployment_id
Example (shape only; use the UI to avoid schema details)
Copy code
{
  "name": "Ensure one run for deployment X",
  "description": "Start a run if none are running",
  "trigger": {
    "type": "metric",
    "posture": "proactive",
    "query": {
      "metric": "flow_runs.running",
      "operator": "<",
      "threshold": 1,
      "where": {
        "deployment_id": "YOUR-DEPLOYMENT-ID"
      },
      "within": "60s",
      "firing_for": "15s"
    }
  },
  "actions": [
    {
      "type": "run-deployment",
      "deployment_id": "YOUR-DEPLOYMENT-ID",
      "parameters": {}
    }
  ],
  "cooldown_seconds": 60
}
Questions for you - Are you on Prefect Cloud or open-source Server? - What deployment (name/ID) should this monitor? - Do you want to ensure exactly one run is active at all times, or at least one (allowing more in parallel)? This will guide whether we add a concurrency limit as well. Happy to draft the exact automation with your deployment details.
b
@Marvin I am using open source server, so then it would not work i think. I would like to have just one parallel.
m
thought for 496.0 seconds
Good news: this works on Prefect 3.x open-source Server too — Automations aren’t Cloud-only. To achieve “keep at least one run alive” while ensuring only one runs at a time: Option A — Automation + Work Pool concurrency (simple, no code changes) 1) Create an Automation in the UI - Trigger: Metric, posture “proactive” - Metric: “Running flow runs” (filter by your specific deployment) - Condition: “< 1” within 30–60s (optionally add a small “firing for” to avoid flapping) - Action: Run deployment (select the same deployment) - Docs: Automations and triggers, CLI management: prefect automation 2) Make sure only one run can execute at once - Put this deployment on a dedicated work pool and cap the pool to 1:
Copy code
prefect work-pool set-concurrency-limit "YOUR_POOL" 1
You can clear it later with:
Copy code
prefect work-pool clear-concurrency-limit "YOUR_POOL"
- CLI reference: Work pool CLI This gives you: - The automation will start a run when none are running. - The work pool concurrency limit ensures you never have more than one run at a time. Option B — In-flow concurrency guard (if you can modify the flow) - Define a global concurrency limit named for this deployment (UI: Concurrency Limits → create, limit=1). - Acquire the slot at the start of your flow:
Copy code
from prefect import flow
  from prefect.concurrency import concurrency

  @flow
  def my_flow():
      with concurrency("my-deployment-one-at-a-time"):
          # your work here
          ...
- Docs: Global concurrency limits Notes - There is no “create automation” CLI; use the UI or API. You can list/inspect/pause/resume via CLI: prefect automation - If you just need a cadence, a schedule is simpler — but it won’t “start only when none are running” (it may queue runs). The automation approach matches your requirement precisely. If you share your deployment name/ID and work pool name, I can outline the exact UI clicks or an API payload you can post to your OSS server to create the automation.
b
@Marvin I think the trigger does not work for opensource as stated over here: https://docs-3.prefect.io/v3/concepts/event-triggers#support-for-event-trigger-types
m
thought for 296.0 seconds
You’re right — thanks for pointing that out. Per the support matrix at Event trigger support, proactive metric triggers (the “no events within X seconds” style) are not supported on open-source Server. So an Automation that fires when there are zero running flow runs won’t work on OSS. Given you want “just one in parallel,” here are two OSS-friendly approaches: Option 1: Schedule + work pool concurrency limit (simple, no code changes) - Create a small-interval schedule on your deployment (e.g., every 30–60s). - Put the deployment on a dedicated work pool with a concurrency cap of 1 so only one run can execute at a time:
Copy code
prefect work-pool set-concurrency-limit "YOUR_POOL" 1
- Work pool CLI docs: prefect work-pool CLI - Result: - If nothing is running, the next scheduled time will start a run. - If one is running, additional runs will wait (not run concurrently). This may create a small backlog if your runtime exceeds the schedule interval. Option 2: Lightweight “watcher” script + work pool concurrency limit - Run a small script (via cron, systemd, or a tiny container) that checks if there are any RUNNING flow runs for your deployment, and if not, starts one. The work pool limit of 1 ensures you never exceed one in parallel even if two checks overlap. Example watcher (sync, suitable for cron)
Copy code
from time import sleep
from uuid import UUID

from prefect.client.orchestration import SyncPrefectClient
from prefect.client.schemas.filters import (
    FlowRunFilter,
    FlowRunFilterDeploymentId,
    FlowRunFilterStateType,
)
from prefect.client.schemas.objects import StateType
from prefect.deployments.flow_runs import run_deployment

DEPLOYMENT_ID = "YOUR-DEPLOYMENT-UUID"  # or use the deployment name if you prefer
CHECK_EVERY_SECONDS = 30

def ensure_one_running():
    with SyncPrefectClient() as client:
        runs = client.read_flow_runs(
            flow_run_filter=FlowRunFilter(
                deployment_id=FlowRunFilterDeploymentId(any_=[UUID(DEPLOYMENT_ID)]),
                state_type=FlowRunFilterStateType(any_=[StateType.RUNNING]),
            ),
            limit=1,
        )
        if not runs:
            # Start a new run. You can also pass parameters or a flow_run_name here.
            run_deployment(UUID(DEPLOYMENT_ID), client=client)

if __name__ == "__main__":
    while True:
        try:
            ensure_one_running()
        except Exception as exc:
            # log and continue; avoid crashing the loop
            print(f"Watcher error: {exc}")
        sleep(CHECK_EVERY_SECONDS)
- Also cap your pool to 1 to enforce single concurrency:
Copy code
prefect work-pool set-concurrency-limit "YOUR_POOL" 1
Docs and references - Event triggers support (why proactive triggers don’t work on OSS): Support for event trigger types - Work pool concurrency: prefect work-pool CLI - Global concurrency limits (optional, if you also want a flow-level lock): Global concurrency limits Would you like me to adapt the watcher for your specific deployment (ID/name) and check cadence? Or would you prefer the schedule + pool-limit approach?