<@ULVA73B9P> Can i have an event based flow with s...
# ask-marvin
l
@Marvin Can i have an event based flow with self-hosted Prefect server and workers?
m
thought for 325.1 seconds
Short answer: Yes. In Prefect 3.x you can run fully event-driven flows with a self-hosted Server and self-hosted workers. The events/automations system is part of OSS; workers just pick up runs created by your automations. Common ways to trigger flows by events: - Prefect events: react to built-in events (e.g., a flow run fails/completes) or to custom events you emit - Custom events from your systems: emit an event to Prefect when something happens, and have an Automation run your deployment - Webhooks: in self-hosted, you’ll typically host a tiny webhook receiver that emits a Prefect event (or directly calls run_deployment) High-level setup 1) Start your self-hosted Server and point your CLI to it - Start server
Copy code
prefect server start
- Set your API URL in env so CLI/SDK/worker use your server
Copy code
export PREFECT_API_URL=<http://127.0.0.1:4200/api>
2) Create a work pool and start a worker (Process, Docker, or Kubernetes) - Create a pool (example: process)
Copy code
prefect work-pool create my-pool --type process
- Start a worker that polls that pool
Copy code
prefect worker start -p my-pool
3) Create and deploy your flow to that pool - In your project folder:
Copy code
prefect deploy ./path/to/flow.py:my_flow -n my-deployment -p my-pool
4) Create an Automation that reacts to an event and runs your deployment - Use the UI (self-hosted UI includes Automations). Pick a trigger, e.g.: - Event trigger on a custom event name you’ll emit (e.g., acme.order.created) - Or built-in events (e.g., when another deployment’s run fails) - Pick the action “Run a deployment” and select your deployment; you can pass parameters (static or derived from event payload). 5) Emit an event from your system when the thing happens - From Python, you can emit a Prefect event directly:
Copy code
from prefect.events import emit_event

  emit_event(
      event="acme.order.created",
      resource={"prefect.resource.id": "acme/orders/123"},
      payload={"order_id": 123, "priority": "high"},
  )
- Alternatively, your webhook receiver (FastAPI/Flask/serverless) can: - emit a Prefect event as above, or - directly start your deployment:
Copy code
from prefect.deployments import run_deployment

    run_deployment(
        name="my-flow/my-deployment",
        parameters={"order_id": 123},
    )
Notes and gotchas - CLI for automations: you can list/inspect/pause/resume/delete via CLI, but creation is typically done in the UI in 3.x. - Workers are just compute; as long as your automation creates a flow run targeting a deployment in the same work pool, your worker will pick it up. - If you want “hosted inbound webhook endpoints” without running your own receiver, that’s a Cloud convenience. In self-hosted, run a tiny receiver that emits a Prefect event or calls run_deployment. - Security: if you expose a webhook, secure it and only allow it to talk to your Prefect API (or have it emit the event via the SDK within your network). Docs to dig deeper - Automations overview and how-to: Creating automations - Event triggers: Event triggers - Webhooks overview: Webhooks Would you like help wiring a specific event source (e.g., GitHub, Stripe, S3, Kafka) into an automation? If you share the event shape/source, I can sketch the concrete trigger and parameter mapping.
l
@Marvin Is is possible to emit the event from the front-end which is react based?
m
thought for 36.7 seconds
Yes, but the recommended way is to go through a tiny backend you control. Options: 1) Recommended: React -> your backend -> Prefect - Your React app POSTs to your backend. - Your backend emits a Prefect event (or directly starts a deployment). This avoids exposing your Prefect Server publicly, handles auth/secrets, and sidesteps browser CORS/WS quirks. Example FastAPI relay that emits a Prefect event:
Copy code
from fastapi import FastAPI, Request
from prefect.events import emit_event

app = FastAPI()

@app.post("/prefect/event")
async def prefect_event(req: Request):
    body = await req.json()
    # Map your frontend payload to a Prefect event
    emit_event(
        event="acme.order.created",
        resource={"prefect.resource.id": f"acme/orders/{body['order_id']}"},
        payload=body,
    )
    return {"ok": True}
Or trigger a deployment directly:
Copy code
from fastapi import FastAPI, Request
from prefect.deployments import run_deployment

app = FastAPI()

@app.post("/prefect/trigger")
async def trigger(req: Request):
    body = await req.json()
    run_deployment(
        name="my-flow/my-deployment",
        parameters={"order_id": body["order_id"]},
    )
    return {"ok": True}
2) Direct from React (possible, but not officially supported) - Prefect events are ingested over a WebSocket endpoint. A browser can open a WS connection to your self-hosted Prefect Server and send an event-shaped JSON message. - Caveats: you must expose your Prefect Server to the browser, handle auth yourself (Cloud requires API keys, OSS has no auth by default), and manage CORS/Origin/WS handshakes. This is why a backend relay is strongly preferred. Minimal sketch of a direct WS send from React (for self-hosted, non-auth):
Copy code
const ws = new WebSocket("<ws://YOUR_PREFECT_SERVER/api/events/out>"); // example shape; endpoint may vary
ws.onopen = () => {
  ws.send(JSON.stringify({
    event: "acme.order.created",
    resource: { "prefect.resource.id": "acme/orders/123" },
    payload: { order_id: 123, source: "frontend" }
  }));
  ws.close();
};
ws.onerror = (e) => console.error("WS error", e);
Note: Prefect does not ship an official browser JS SDK for events; the WS path/shape is internal and may change. Use at your own risk. Security considerations - Do not expose your self-hosted Prefect API broadly. Anyone who can reach it could emit events and influence automations. - If you must go direct from the browser, put Prefect behind a reverse proxy that enforces auth and strict Origin rules. - The backend relay pattern is the safest and most stable. Docs - Events and automations overview: Event triggers and Creating automations - Webhooks discussion (and why backends help): Webhooks Which route do you prefer? If you share your constraints (e.g., can you run a tiny backend, is your Prefect Server reachable from the client, any auth requirements), I can tailor a working example.
n
you don't need to catch the event yourself @Marvin, there are automations in OSS. deployment trigger
expect
ing
acme.order.created
would suffice
m
thought for 38.8 seconds
Totally—you're right. In OSS (Prefect 3.x), you don’t need to “catch” the event yourself in code. If you create an Automation with a Deployment Event Trigger that expects
acme.order.created
, Prefect will listen for that event and run your deployment automatically; your self-hosted worker will pick up the run. The one thing you still need is a way to emit that event into Prefect: - Recommended: React -> your backend -> Prefect. Your backend emits the event (or runs the deployment) when the user action occurs in the UI. - Possible but not recommended: emit directly from the browser to your Prefect Server’s events websocket. This requires exposing your Prefect API to the browser and handling auth/CORS/WS details. Once the event is emitted, Automations will handle the rest—no custom “listener” service required. If helpful, I can share: - A quick UI walkthrough for creating the “Event name equals acme.order.created” trigger that runs your deployment and maps parameters from the event payload - A minimal backend relay example you can call from React that emits
acme.order.created
References: - Automations: Creating automations - Event triggers: Event triggers - Webhooks (why a tiny backend relay helps): Webhooks
upvote 1