<@ULVA73B9P> does self-hosted prefect have the con...
# ask-marvin
c
@Marvin does self-hosted prefect have the concept of admin and non admin users? It seems that there is no such thing, even if I add an admin string config. I cannot use prefect-cloud. How can I see the logged in user in the UI, and limit the amount of operations certain users can do, and logout all from UI? It seems that a reverse-proxy is not a good solution, with those requirements. Is there a way to make reverse-proxy work that way with OICD? Would fastapi middleware work?
m
thought for 142.2 seconds
Short answer: in self‑hosted Prefect Server there is no concept of users, sessions, admins, or RBAC. The UI has no login/logout and there is no “logged‑in user” to display. Details and options: - The
PREFECT_API_AUTH_STRING
setting is a single shared HTTP Basic credential (e.g.,
user:pass
) that protects the API. It does not create per‑user accounts or roles, and it won’t make the UI show a user identity. - Docs: Security settings (Prefect 3) - Because the open‑source server has no built‑in identity or roles: - You cannot see the logged‑in user in the UI (there isn’t one). - You cannot limit specific operations per user from within Prefect itself. - You cannot “log out all users” from the UI (there’s no session concept). What you can do self‑hosted - Put Prefect behind an OIDC/SAML auth proxy (e.g., oauth2‑proxy with NGINX or Traefik). This gives you: - Login via your IdP before anyone can reach the UI/API. - Optionally, coarse‑grained “RBAC” at the proxy by HTTP method/path (e.g., allow GETs to everyone, require “prefect-admin” group for POST/PUT/PATCH/DELETE). - A central logout endpoint (handled by the proxy/IdP), which effectively “logs out” everyone by revoking/expiring sessions. - Caveats: - Prefect Server will not consume identity headers or enforce per‑resource permissions; all enforcement is at the proxy. - You can approximate roles (viewer vs admin) by method/path rules, but not granularly (e.g., “this user can only manage these deployments” isn’t possible in OSS). A sketch of the proxy approach - Require auth for both the UI and API routes so the SPA’s API calls are also authenticated. - Use proxy features to differentiate read vs write: - Example idea (NGINX + oauth2‑proxy): allow all authenticated users to GET
/
and `/api/*`; require membership in an “admin” group (validated by oauth2‑proxy) for POST/PUT/PATCH/DELETE to
/api/*
. - Provide a logout link that calls the proxy’s logout endpoint (which clears cookies and/or redirects to the IdP logout). Community example (reverse proxy around Server): - Docker Compose with a reverse proxy example (community example you can adapt) On FastAPI middleware - Prefect Server is built on FastAPI, so in theory you could fork/wrap it with custom middleware. We don’t recommend this for production: you’ll own the maintenance, and upgrades can break your patches. A fronting auth proxy is the supported, maintainable pattern. If you need first‑class RBAC and user management - Those features (roles, workspace permissions, audit logs, etc.) are part of Prefect Cloud. Since you can’t use Cloud, the proxy approach above is your best path for self‑hosted. - Cloud user management docs (for reference only): Manage users (Cloud) Questions to tailor a concrete setup for you - How are you running Server (Docker Compose, Kubernetes, something else)? - Which IdP do you use (Okta, Azure AD, Keycloak, …)? - Which operations do you want to allow vs restrict (e.g., “view only” vs “create/delete deployments/runs”)? If you share those, I can draft a minimal NGINX + oauth2‑proxy (or Traefik) config that enforces: - Auth for all routes - Group‑based access for write methods to the API - A logout link that invalidates all sessions via your IdP
c
@Marvin assuming I am using a reverse proxy is there a way, from a flow and task execution to access the logged in username? I need to keep track of things like "user x executed flow y"
m
thought for 325.4 seconds
Short answer: not automatically. In self‑hosted Prefect Server there’s no built‑in user/session, so flows and tasks cannot “see” a logged‑in user unless you pass that identity into the flow run when you create it. A workable pattern behind a reverse proxy (OIDC or similar): - Have the proxy (e.g., oauth2‑proxy + NGINX/Traefik) authenticate the user and expose their identity via headers like
X-Auth-Request-User
or
X-Forwarded-User
. - Create a tiny gateway endpoint (e.g., FastAPI) that reads that header and calls the Prefect API to create the flow run, injecting the username into: - parameters (e.g.,
triggered_by
), and/or - tags (e.g.,
user:<username>
), and/or - context (e.g.,
{"actor": "<username>"}
) Notes: - Prefect OSS won’t populate this for you; you must inject it at run creation. - If you trigger runs via the built‑in UI “Run” button, Prefect won’t capture the user. Proxies generally don’t modify JSON request bodies safely; so if you need identity attached, start runs via your gateway instead of the default UI. Example: calling the API directly
Copy code
POST /api/deployments/<deployment_id>/create_flow_run
Content-Type: application/json
Authorization: Basic <...>  # if using PREFECT_API_AUTH_STRING

{
  "name": "my-run",
  "parameters": { "triggered_by": "<mailto:alice@example.com|alice@example.com>" },
  "tags": ["user:alice@example.com"],
  "context": { "actor": "<mailto:alice@example.com|alice@example.com>" }
}
Reading the user inside flows and tasks (Prefect 3.x)
Copy code
from prefect import flow, task, get_run_context

def get_triggering_user():
    ctx = get_run_context()
    # Prefer a parameter
    if "triggered_by" in ctx.parameters:
        return ctx.parameters["triggered_by"]
    # Or read from flow run context
    if getattr(ctx, "flow_run", None) and ctx.flow_run.context:
        user = ctx.flow_run.context.get("actor")
        if user:
            return user
    # Or parse a tag if you added one like "user:<name>"
    if getattr(ctx, "flow_run", None) and ctx.flow_run.tags:
        for t in ctx.flow_run.tags:
            if t.startswith("user:"):
                return t.split(":", 1)[1]
    return "unknown"

@task
def do_work():
    user = get_triggering_user()
    print(f"Task running; triggered by {user}")

@flow
def my_flow():
    user = get_triggering_user()
    print(f"Flow running; triggered by {user}")
    do_work()
Optional: record it as an artifact for visibility in the UI
Copy code
from prefect.artifacts import create_markdown_artifact
create_markdown_artifact(f"Run requested by {get_triggering_user()}")
Why a gateway instead of only a reverse proxy? - The proxy authenticates but Prefect doesn’t read identity headers or enforce roles. A small gateway lets you turn identity into run metadata deterministically. Docs (for reference on auth basics in Server): Security settings (Prefect 3) If you share your proxy/IdP (e.g., oauth2‑proxy + NGINX, Traefik, Keycloak, etc.), I can sketch a minimal gateway route that: - extracts
X-Auth-Request-User
, - calls Prefect’s create‑flow‑run endpoint, - adds
triggered_by
and a
user:<name>
tag, - returns the created run ID and a link to the run.
c
@Marvin what about if I used prefect cloud? how can I get from within a flow/task the info on who triggered the operation?
m
thought for 73.8 seconds
In Prefect Cloud, runs are associated with users and automations for auditing and RBAC, but that identity is not injected into the flow/task runtime context. So inside a flow/task there isn’t a built‑in variable like “triggered_by_user”. What you can do instead Option A (recommended, works in Cloud and OSS) - Inject the actor at flow run creation time (when calling the API or CLI), then read it inside the flow: - Put it in parameters (e.g.,
triggered_by
) - Or add a tag like
user:alice@example.com
- Or add to the run
context
field Inside your flow/task:
Copy code
from prefect import flow, task, get_run_context

def get_actor_from_runtime():
    ctx = get_run_context()

    # Prefer a parameter
    if "triggered_by" in ctx.parameters:
        return ctx.parameters["triggered_by"]

    # Or read from flow run context (if you put it there at creation)
    fr = getattr(ctx, "flow_run", None)
    if fr and getattr(fr, "context", None):
        actor = fr.context.get("actor")
        if actor:
            return actor

    # Or parse a tag you added like "user:<name>"
    if fr and fr.tags:
        for t in fr.tags:
            if t.startswith("user:"):
                return t.split(":", 1)[1]

    return None

@task
def do_work():
    actor = get_actor_from_runtime() or "unknown"
    print(f"Task running; triggered by {actor}")

@flow
def my_flow():
    actor = get_actor_from_runtime() or "unknown"
    print(f"Flow running; triggered by {actor}")
    do_work()
- If you launch runs via API/CLI, you can set these values at creation time. For example (API shape):
Copy code
POST /api/deployments/<deployment_id>/create_flow_run
{
  "name": "run-with-actor",
  "parameters": {"triggered_by": "<mailto:alice@example.com|alice@example.com>"},
  "tags": ["user:alice@example.com"],
  "context": {"actor": "<mailto:alice@example.com|alice@example.com>"}
}
Option B (Cloud‑only): Look up the actor via audit logs/events - Prefect Cloud records who triggered a run (user, automation, system), and shows it in the UI and audit logs. - From within your flow, you can: 1) Get the current flow run ID (
get_run_context().flow_run.id
) 2) Query Cloud’s audit logs/events for the “flow run created” event where the resource matches that ID 3) Read the event’s actor (user/automation) and use it in your code Docs for Cloud audit logs/events: Audit logs (Cloud) Notes and tradeoffs - The UI “Run” button does not inject identity into the run’s parameters/tags/context; the identity only appears in Cloud’s audit logs. If you need actor info inside your code, Option A is most reliable. - Automations will show as the actor when they trigger runs; you can still tag/contextualize runs at creation if you build automations or webhooks that include metadata. - If you want, I can provide a snippet that uses your Prefect Cloud API key to query the audit log/events for the current run and extract the actor—just let me know your preferred language and any constraints.
c
@Marvin with a self-hosted solution and a reverse proxy that does the authenticaiton and exposes to prefect the user logged in with a X_AUTHORISED_USER HTTP header, how can I create an event, emitted when a flow run is requested, that contains the information about the user and associate it to the run flow? e.g. an evant that says "User x requested run y" and how can I can query the events within the flow and get the same information
m
thought for 219.7 seconds
Yes—doable with self-hosted + reverse proxy, but you need to inject the username yourself and emit the event explicitly. Prefect Server won’t read
X_AUTHORISED_USER
on its own. Recommended pattern - Put a small “trigger” gateway behind your proxy. It: 1) Reads
X_AUTHORISED_USER
2) Creates the flow run via the Prefect API 3) Emits an event tied to that flow run with the username in the payload 4) Optionally injects the username into the run as a parameter/tag/context so your flow can read it without querying events Key API/docs - Create events: /events/create - Read events: /events/read - Python helper to emit events: prefect.events.emit_event Example: gateway endpoint (FastAPI-style)
Copy code
import os
import httpx
from fastapi import FastAPI, Request, HTTPException
from prefect.events import emit_event

PREFECT_API_URL = os.environ["PREFECT_API_URL"]
# If you're using PREFECT_API_AUTH_STRING (Basic Auth), expose it as an env var
PREFECT_API_AUTH = os.environ.get("PREFECT_API_AUTH")  # e.g. "Basic <base64>"

app = FastAPI()

@app.post("/trigger/{deployment_id}")
async def trigger_flow(deployment_id: str, request: Request):
    user = request.headers.get("X_AUTHORISED_USER")
    if not user:
        raise HTTPException(status_code=401, detail="Missing X_AUTHORISED_USER")

    # 1) Create the flow run and attach user info so the flow can read it directly
    create_url = f"{PREFECT_API_URL}/api/deployments/{deployment_id}/create_flow_run"
    body = {
        "name": f"requested-by-{user}",
        "parameters": {"triggered_by": user},           # easy to read inside the flow
        "tags": [f"user:{user}"],                       # useful for filtering
        "context": {"actor": user},                     # optional
    }
    headers = {"Authorization": PREFECT_API_AUTH} if PREFECT_API_AUTH else {}
    async with httpx.AsyncClient(timeout=20) as client:
        resp = await <http://client.post|client.post>(create_url, json=body, headers=headers)
        resp.raise_for_status()
        fr = resp.json()
        flow_run_id = fr["id"]

    # 2) Emit an event associated with the flow run
    # Resource id format is important to associate to the flow run:
    #   "prefect.flow-run.<UUID>"
    flow_run_resource = {"prefect.resource.id": f"prefect.flow-run.{flow_run_id}"}
    user_resource = {"prefect.resource.id": f"myorg.user.{user}"}  # your own namespace

    emit_event(
        event="myorg.flow-run.requested",
        resource=flow_run_resource,
        related=[user_resource],
        payload={"user": user},
    )

    return {"flow_run_id": flow_run_id}
Inside your flow: read the user directly (simplest) - Since the gateway added user info to the run, you can read it without hitting the events API:
Copy code
from prefect import flow, task, get_run_context

def get_actor_from_runtime():
    ctx = get_run_context()
    if "triggered_by" in ctx.parameters:
        return ctx.parameters["triggered_by"]
    fr = getattr(ctx, "flow_run", None)
    if fr and getattr(fr, "tags", None):
        for t in fr.tags:
            if t.startswith("user:"):
                return t.split(":", 1)[1]
    # if you set context={"actor": "..."} at creation time:
    if fr and getattr(fr, "context", None):
        return fr.context.get("actor")
    return None

@task
def do_work():
    user = get_actor_from_runtime() or "unknown"
    print(f"Task running; triggered by {user}")

@flow
def my_flow():
    user = get_actor_from_runtime() or "unknown"
    print(f"Flow running; triggered by {user}")
    do_work()
Inside your flow: query the event you emitted - If you specifically want to query the event, call the read-events REST API and filter by: - event name:
myorg.flow-run.requested
- resource:
prefect.flow-run.<current_run_id>
Skeleton example (shape depends on the filter JSON your server expects; see docs):
Copy code
import os
import httpx
from prefect.runtime import flow_run

def get_requesting_user_from_event():
    api = os.environ["PREFECT_API_URL"]
    auth = os.environ.get("PREFECT_API_AUTH")  # e.g., "Basic <...>"
    headers = {"Authorization": auth} if auth else {}

    body = {
        "limit": 1,
        "order": "DESC",
        "filter": {
            "name": {"any_": ["myorg.flow-run.requested"]},
            "resource": {"any_": [{"id": f"prefect.flow-run.{flow_run.id}"}]}
        }
    }

    r = <http://httpx.post|httpx.post>(f"{api}/api/events/read", json=body, headers=headers, timeout=20)
    r.raise_for_status()
    events = r.json().get("events", [])
    if not events:
        return None
    return events[0].get("payload", {}).get("user")
Notes and tips - Associating the event to the flow run: use
resource={"prefect.resource.id": f"prefect.flow-run.{flow_run_id}"}
when emitting. This is how Prefect links events to runs. - Prefer “inject at creation” for runtime use: if your flow logic needs the actor, putting it in parameters/tags/context is the most reliable and avoids extra API calls. - Ensure your proxy allows and forwards the
/api/events/*
endpoints to Prefect Server (they’re needed for emit/read). - You can add more related resources (deployment, workspace, etc.) if helpful. The event schema supports a list of
related
resources. Docs again for quick reference: - Create events: /events/create - Read events: /events/read - Python helper to emit events: prefect.events.emit_event If you share your proxy stack (oauth2-proxy/Keycloak/NGINX/Traefik) I can tailor the gateway route and headers exactly to your environment.
c
@Marvin is there a specific design reason for which prefect does not ready information passed by for examples reverse proxies? It seems that the OSS version lacks a fundamental part. why is it so?
m
thought for 28.2 seconds
Great question. The short version is: Prefect OSS’s Server is intentionally scoped as an orchestration API and scheduler without a built‑in identity system. That’s why it does not read reverse‑proxy auth headers or implement users/RBAC. Design reasons and tradeoffs - Clear scope and portability - OSS Server is designed to be easy to run anywhere (Docker, K8s, air‑gapped, local dev) with minimal moving parts. Adding users, sessions, RBAC, and IdP integrations would introduce stateful auth, migrations, policy engines, and long‑term maintenance overhead. - Security boundary at your infra - Many self‑hosted users deploy in private networks and front Prefect with their existing identity stack (NGINX/Traefik + oauth2‑proxy, Keycloak, Okta, Azure AD). Prefect’s guidance is to enforce auth at the network/proxy layer, then let Prefect focus on orchestration. - Trusting arbitrary headers is dangerous unless you also ship a robust “trust proxy” model (TLS termination rules, allowed upstreams, header normalization, spoofing protection). That becomes a security product in itself. - “Reading headers” isn’t enough for RBAC - Even if Server read
X-Authenticated-User
, you’d still need a full role model, permissions for each route, UI affordances, audit logs, group mapping, etc. Prefect Cloud implements that end‑to‑end; OSS avoids duplicating that complexity. - Consistency and maintenance - Every IdP/proxy emits different headers and claims. Standardizing and supporting these across versions (including UI behavior, logouts, token lifecycles) would add significant surface area and breakage risk for self‑hosted users. What OSS provides instead - Minimal API protection knobs (e.g.,
PREFECT_API_AUTH_STRING
) and strong recommendation to place Prefect behind your proxy/IdP. - Composability: you can inject identity at run creation (parameters/tags/context) and/or emit/query events for audit‑like trails. This keeps Prefect OSS flexible while letting you use the auth components you already standardize on. If you’re looking for first‑class identity, permissions, and audit within the product, those are features of Prefect Cloud by design, where we can deliver a cohesive experience across UI/API/workspaces and maintain it over time. References - OSS security settings: Security settings (Prefect 3) - Security and shared responsibility: Prefect Security and Shared Responsibility Model If you want, I’m happy to help you harden your reverse‑proxy + small “trigger gateway” pattern so you can attach users to runs and emit/query events reliably in your self‑hosted environment.