Rasmus Simmelkjær Berntsen
11/03/2025, 5:35 AMMarvin
11/03/2025, 5:35 AMRasmus Simmelkjær Berntsen
11/03/2025, 5:39 AMMarvin
11/03/2025, 5:39 AMMarvin
11/03/2025, 5:39 AMRasmus Simmelkjær Berntsen
11/03/2025, 5:39 AMMarvin
11/03/2025, 5:40 AMMarvin
11/03/2025, 5:40 AMprefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
- If you run the command from inside a Docker container on the same network as your server, use:
export PREFECT_API_URL=<http://prefect-server:4200/api>
(replace prefect-server with your service name)
2) Register built-in Prefect block types
Run these once; they’ll appear in the UI afterward.
prefect block register -m prefect.blocks.system
prefect block register -m prefect.blocks.notifications
prefect block register -m prefect.blocks.webhook
Verify:
prefect block type ls
3) Register integration block types (AWS/GCP/Docker/etc.)
Install the integration package in the environment where you run the CLI, then register:
# AWS
pip install prefect-aws
prefect block register -m prefect_aws
# GCP
pip install prefect-gcp
prefect block register -m prefect_gcp
# Docker
pip install prefect-docker
prefect block register -m prefect_docker
4) Create block instances (via UI or code)
- From the CLI (opens a link to the UI to create a block):
prefect block create
- Or programmatically:
from prefect.blocks.system import Secret
Secret(value="super-secret").save("my-secret", overwrite=True)
List configured block instances:
prefect block ls
Notes:
- Block types and instances are stored in your server’s database. If you reset the DB, you’ll need to re-register.
- You only need the integration package installed where you run prefect block register and anywhere you use that block in code (e.g., workers).
Helpful docs:
- Blocks in Prefect 3: Concepts: Blocks
- Self-hosted server with docker-compose: Self-hosted: docker-compose
If you tell me which blocks you need (e.g., S3, GCS, Docker image, Slack), I can give you the exact module(s) to register.Rasmus Simmelkjær Berntsen
11/03/2025, 5:44 AMMarvin
11/03/2025, 5:44 AMMarvin
11/03/2025, 5:45 AMprefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
- If you’re running this inside a Docker container on the same network as your server, use:
export PREFECT_API_URL=<http://prefect-server:4200/api>
2) Register the Notifications block types
prefect block register -m prefect.blocks.notifications
Confirm they’re available:
prefect block type ls
# (optionally filter)
# prefect block type ls | grep notifications
3) Create a notification block instance
- From the CLI (opens a UI link). Find the slug from the previous step and create it, e.g.:
prefect block create notifications/slack-webhook
# or notifications/microsoft-teams-webhook, notifications/sendgrid-email, etc.
- Or programmatically:
from prefect.blocks.notifications import SlackWebhook
SlackWebhook(
url="<https://hooks.slack.com/services/XXX/YYY/ZZZ>"
).save("my-slack", overwrite=True)
4) Use it in a flow (quick test)
from prefect import flow
from prefect.blocks.notifications import SlackWebhook
@flow
def demo():
SlackWebhook.load("my-slack").notify("Hello from Prefect!")
demo()
Docs for reference:
- Blocks in Prefect 3: Concepts: Blocks
- Self-hosted docker-compose: Self-hosted: docker-compose
Which notifier do you want to set up (Slack, Teams, email, etc.)? I can share the exact slug and a tailored example.Marvin
11/03/2025, 5:46 AMprefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
- If you’re running the CLI inside a Docker container on the same compose network, point to the API service name, e.g.:
prefect config set PREFECT_API_URL=<http://prefect-server:4200/api>
(adjust service name/port to your compose file; sometimes host.docker.internal works too)
2) Register core Prefect block types
Run these (separately) to register built-ins like Secrets, Notifications, Filesystems, and Webhooks:
prefect block register -m prefect.blocks.system
prefect block register -m prefect.blocks.notifications
prefect block register -m prefect.blocks.webhook
prefect block register -m prefect.filesystems
3) Install and register integration blocks (optional)
Install the integration package in the same environment where you’re running the prefect CLI, then register its blocks. Examples:
- AWS
pip install prefect-aws
prefect block register -m prefect_aws
- GCP
pip install prefect-gcp
prefect block register -m prefect_gcp
- Docker
pip install prefect-docker
prefect block register -m prefect_docker
- Kubernetes
pip install prefect-kubernetes
prefect block register -m prefect_kubernetes
- Azure
pip install prefect-azure
prefect block register -m prefect_azure
4) Verify registration
- See available block types on your server:
prefect block type ls
- See configured block instances (will be empty until you create them):
prefect block ls
5) Create block instances
- From the CLI (opens a link to your UI):
prefect block create
- Or create them directly in the UI under Blocks.
Quick check: Are you running the prefect CLI on your host or inside one of your Docker containers? If it’s inside Docker, I can help you set the correct PREFECT_API_URL for your compose setup.Rasmus Simmelkjær Berntsen
11/03/2025, 6:08 AMMarvin
11/03/2025, 6:08 AMMarvin
11/03/2025, 6:10 AMprefect.runtime helpers inside your flow run:
- Deployment name: prefect.runtime.deployment.name (None if not running from a deployment)
- Flow-run tags: prefect.runtime.flow_run.tags (a set of strings)
Example: build a custom failure message and send via a SlackWebhook block:
from prefect import flow
from prefect.blocks.notifications import SlackWebhook
from prefect.runtime import flow_run, deployment
def notify_failure(err: Exception):
dep_name = deployment.name or "no-deployment"
tags = ", ".join(sorted(flow_run.tags or []))
msg = (
f"❌ Flow failed\n"
f"- Flow: {flow_run.flow_name}\n"
f"- Run: {flow_run.name}\n"
f"- Deployment: {dep_name}\n"
f"- Tags: [{tags}]\n"
f"- Details: {flow_run.ui_url}\n"
f"- Error: {err}"
)
SlackWebhook.load("my-slack").notify(msg)
@flow
def my_flow():
try:
# your logic here
raise RuntimeError("Boom!") # demo
except Exception as e:
notify_failure(e)
raise
Notes:
- deployment.name will be None for ad‑hoc runs (not from a deployment).
- flow_run.tags includes any tags inherited from the deployment plus any added at run time.
Are you sending notifications via code (like above) or with an Automation in the UI? If you’re using Automations, I can share a Jinja template that references the same fields (deployment name, tags, run link) directly in the message.Rasmus Simmelkjær Berntsen
11/03/2025, 7:04 AMMarvin
11/03/2025, 7:04 AMMarvin
11/03/2025, 7:08 AMkubectl get pods -n <your-namespace> | grep prefect
- In the services pod logs you should see messages like “Event persister started”, “Reactive triggers service started”, “Actions service started”.
kubectl logs deploy/<your-prefect-services-deployment> -n <ns> --tail=200
If you don’t have a services deployment, events won’t be persisted and automations won’t fire. If you installed via Helm, ensure the “services” component is enabled in your values.
2) Confirm your CLI points at the same API/workspace
From your workstation (or a pod with network to the API):
prefect config set PREFECT_API_URL=http://<your-prefect-api-service>:4200/api
prefect automation ls
You should see your automation listed and enabled. If not, you’re likely pointed at another workspace.
3) Stream events to confirm they’re being emitted
In one terminal, subscribe to events:
prefect event stream --format text
Then kick off a flow run that changes state (e.g., fail a run). You should see events like:
- prefect.flow-run.Running
- prefect.flow-run.Completed or prefect.flow-run.Failed
If you see no events here, focus on step 1 (services) and server logs. If you do see events, move to step 4.
4) Double-check your automation configuration
- Make sure it’s Enabled (not Paused).
- Trigger: If it’s based on flow run state, ensure the filter matches the actual events you saw in step 3.
- Common approach: filter on the resource type “flow run” and state “Failed/Completed/Running” instead of hard-coding event names.
- Scope filters: If you limited to a specific deployment/flow/work pool/tag, confirm those match your runs.
- Posture: If it’s “proactive” (fires when events are absent), it won’t fire immediately; reactive triggers fire on matching events.
Use:
prefect automation inspect <automation-id>
to review the full config and confirm filters.
5) Test the Microsoft Teams action and network connectivity
- Ensure your Teams webhook block exists and is referenced by the automation.
- From a simple flow, send a test message using the block (this tests your webhook URL, but from the worker network):
from prefect.blocks.notifications import MicrosoftTeamsWebhook
MicrosoftTeamsWebhook.load("my-teams").notify("Test from flow")
- If that works but automations still don’t send, the Actions service may lack egress to the internet. Try a “Call webhook” action in an automation to https://webhook.site/<your-id> and see if it hits; if not, allow egress from the services pod.
6) Useful commands
- List and inspect automations:
prefect automation ls
prefect automation inspect <id>
- Stream events in real time:
prefect event stream --format text
Docs for reference:
- Events and automations: Concepts: Events and How-to: Creating automations
- Self-hosted/server on K8s (Helm): Helm: Prefect Server
If you can share:
- How you installed Prefect on K8s (Helm chart and values)
- Output of prefect event stream while you start a flow run
- Whether your Teams block works from inside a flowMarvin
11/03/2025, 7:08 AM