<@ULVA73B9P> if I use modal to run my flow with pr...
# ask-marvin
s
@Marvin if I use modal to run my flow with prefect sdk installed. Does my workflow execution will be tracked in prefect UI. I don't have any work pools configured.
m
thought for 205.6 seconds
Short answer: Yes. In Prefect 3.x, if your flow runs inside Modal with the Prefect SDK installed and it can reach your Prefect Cloud (or a publicly reachable Prefect Server), the run will be fully tracked in the Prefect UI — even without any work pools or deployments. What you need - Set these environment variables in the Modal runtime: -
PREFECT_API_URL
(e.g., `https://api.prefect.cloud/api/accounts/&lt;ACCOUNT_ID&gt;/workspaces/&lt;WORKSPACE_ID&gt;`) -
PREFECT_API_KEY
- Ensure
prefect
is installed in the Modal image/environment. - Just call your flow function (e.g.,
my_flow()
); Prefect will create and report a flow run. Minimal example (conceptual)
Copy code
# In your Modal function/container:
export PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>>"
export PREFECT_API_KEY="<YOUR_KEY>"

# Your code:
from prefect import flow, get_run_logger

@flow(log_prints=True)
def my_flow():
    print("Hello from Modal!")  # will appear in UI with log_prints=True
    get_run_logger().info("Tracked log")

if __name__ == "__main__":
    my_flow()  # creates a tracked flow run in the UI
What you’ll see in the UI - The run will show up under Flow Runs with state, logs, and task runs. - It won’t be associated with a deployment/work pool (that’s expected). Limitations without work pools/deployments - No scheduling from the UI (cron/interval/etc.). For scheduling or “Run” from the UI, create a deployment. - No work-pool based queuing/worker execution or concurrency limits. - Your local Prefect Server won’t work unless it’s reachable from Modal; Prefect Cloud is easiest for remote compute. If/when you want scheduling or to route runs to Modal from Prefect, use a Modal work pool and create a deployment. Docs and references - Connect your code to Cloud (API URL and key): Connect to Prefect Cloud - Manage settings via env vars: Manage Prefect settings - Logging in flows/tasks: Add logging to your flows - Scheduling (requires deployments): Scheduling concepts - Using Modal with Prefect (work pools/deployments): Run with Modal If you can share whether you’re on Prefect Cloud or a self-hosted Server, I can suggest the exact
PREFECT_API_URL
to use and a Modal-friendly way to set those env vars.
s
@Marvin how can I try my flow executions running on modal better without deployment so I can pivot to ui to why this alertid is not processed
m
thought for 259.9 seconds
Great question. You can keep running your flow on Modal without a deployment and still make each run easy to find and debug in the Prefect UI by “stamping” runs with your alert_id and sending structured logs. Here’s a practical setup that works well in Prefect 3.x: What to add to your flow - Make alert_id a flow parameter. - Name the run with alert_id so it’s easy to find. - Add tags including alert_id (so you can filter in the UI). - Log with alert_id in every message and capture print statements. Example
Copy code
from prefect import flow, get_run_logger, tags

@flow(flow_run_name=lambda alert_id: f"alert-{alert_id}", log_prints=True)
def process_alert(alert_id: str):
    logger = get_run_logger(alert_id=alert_id)  # adds alert_id as structured field
    <http://logger.info|logger.info>("Starting alert processing")
    print(f"Processing alert {alert_id}")        # will be sent to UI due to log_prints=True
    # ... your logic here ...
    <http://logger.info|logger.info>("Finished alert processing")


if __name__ == "__main__":
    # local test
    with tags("source:modal", "env:dev", lambda: f"alert:{'12345'}"):
        process_alert(alert_id="12345")
Running from Modal (no deployment) - Ensure your Modal runtime has Prefect and Prefect Cloud creds. - Wrap the flow call with tags that include the alert id. Example sketch (adapt to your Modal setup):
Copy code
import modal
from prefect import tags
from my_flows import process_alert

image = modal.Image.debian_slim().pip_install("prefect")

stub = modal.Stub("alert-processor")

# Store PREFECT_API_URL and PREFECT_API_KEY in a Modal secret named "prefect"
@stub.function(image=image, secrets=[modal.Secret.from_name("prefect")])
def run_flow(alert_id: str):
    with tags("source:modal", f"alert:{alert_id}"):
        process_alert(alert_id=alert_id)
How to pivot in the UI by alert_id - Flow runs list: - Filter by Tags → select
alert:<id>
or
source:modal
- Or search by Run name like
alert-<id>
- Logs: - Open the run and use the text search to find the
alert_id
value - Because we used
get_run_logger(alert_id=...)
, the log records include the alert_id as structured metadata, and
log_prints=True
will capture print output too Helpful extras - Turn up logging for deeper debugging: - Set
PREFECT_LOGGING_LEVEL=DEBUG
in your Modal environment - Create a small artifact summarizing the outcome for quick scanning (optional):
Copy code
from prefect.artifacts import create_markdown_artifact

  create_markdown_artifact(
      markdown=f"### Alert {alert_id}\nStatus: Processed successfully",
  )
- Ensure Modal can talk to Prefect Cloud: - Set
PREFECT_API_URL=<https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>>
- Set
PREFECT_API_KEY=<YOUR_KEY>
Docs and references - Connect your code to Cloud (API URL and Key): Connect to Prefect Cloud - Manage Prefect settings (env vars like PREFECT_LOGGING_LEVEL): Manage Prefect settings - Logging in flows/tasks (log_prints, get_run_logger): Add logging to your flows - Modal + Prefect overview (for when you want to move to deployments later): Run with Modal Questions to tailor further - Are you on Prefect Cloud or self-hosted Server? - Do you run one alert per flow run (recommended) or batch multiple alert_ids? - Do you already see your Modal-triggered runs in the UI? If not, I can help verify your
PREFECT_API_URL
and
PREFECT_API_KEY
setup.
s
@Marvin how to give name to flow in code so I can use variable
m
thought for 204.8 seconds
You can name each flow run from code in Prefect 3.x using the
flow_run_name
option on the
@flow
decorator. The easiest way is to reference your flow parameters directly. Use a flow parameter in the name
Copy code
from prefect import flow

@flow(flow_run_name="alert-{alert_id}")
def process_alert(alert_id: str):
    ...
- Parameter names in
{}
are substituted at runtime, so calling
process_alert(alert_id="12345")
creates a run named
alert-12345
. Multiple variables
Copy code
@flow(flow_run_name="modal-{env}-alert-{alert_id}")
def process_alert(alert_id: str, env: str = "dev"):
    ...
Dynamic name via callable (no args)
Copy code
from datetime import datetime
from prefect import flow

@flow(flow_run_name=lambda: f"alert-run-{datetime.utcnow().isoformat()}")
def process_alert(alert_id: str):
    ...
Programmatically set/override the name inside the flow (for full control)
Copy code
from prefect import flow
from prefect.context import get_run_context
from prefect.client.orchestration import SyncPrefectClient

@flow
def process_alert(alert_id: str):
    run_id = get_run_context().flow_run.id
    with SyncPrefectClient() as client:
        client.set_flow_run_name(run_id, f"alert-{alert_id}-manual")
    ...
Notes - These work without a deployment; the run will appear in the UI with your custom name. - If you use the template form, make sure the placeholders match your parameter names exactly. Docs: Custom metadata (naming runs, tags, etc.)
s
@Marvin is the Prefect cloud api domain customisable?
m
thought for 100.3 seconds
Short answer: Not today. For Prefect Cloud, the API domain is fixed at https://api.prefect.cloud. What you can customize - The full API URL you point your code to is: - Cloud:
<https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>>
- Self-hosted Server: whatever host you run (e.g., `http://127.0.0.1:4200/api`) - Set this via
PREFECT_API_URL
(and
PREFECT_API_KEY
for Cloud). Notes - Prefect Cloud is a multi-tenant service exposed at
api.prefect.cloud
. There aren’t regional/custom domains. - Enterprise networking options like PrivateLink exist for restricted network paths, but they don’t change the public API domain. Docs - Connect your code to Cloud: Connect to Prefect Cloud - Settings reference: Manage Prefect settings - PrivateLink (Enterprise): Secure access by PrivateLink
s
@Marvin how to use prefect decorators in modal function which one to keep above and below for tasks, flow and app
m
thought for 38.1 seconds
Great question — decorator order matters. Recommended pattern (don’t stack them) - Keep Prefect decorators on your business logic. - Tasks: only
@task
- Flow: only
@flow
- Put the Modal decorator on a thin entrypoint function that just calls your flow. Example
Copy code
import modal
from prefect import flow, task, tags
from prefect.logging import get_run_logger

app = <http://modal.App|modal.App>("alert-processor")

@task
def do_work(x: int) -> int:
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"working on {x}")
    return x * 2

@flow(flow_run_name="alert-{alert_id}", log_prints=True)
def process_alert(alert_id: str):
    with tags("source:modal", f"alert:{alert_id}"):
        return do_work(alert_id)

# Modal entrypoint (no Prefect decorators here)
@app.function(image=modal.Image.debian_slim().pip_install("prefect"), secrets=[modal.Secret.from_name("prefect")])
def run_flow(alert_id: str):
    # Ensure PREFECT_API_URL and PREFECT_API_KEY are available via the Modal secret
    process_alert(alert_id)
Why this is best - Your flow and tasks run inside the Modal container and are fully tracked in Prefect. - You avoid decorator conflicts and keep concerns separate (Modal = where to run, Prefect = how to orchestrate). If you must stack on the same function - Put the Modal decorator OUTERMOST and the Prefect decorator directly above the function. - Correct:
Copy code
@app.function(...)
  @flow(...)
  def main(...):
      ...
- Avoid the reverse:
Copy code
@flow(...)
  @app.function(...)
  def main(...):
      ...
This starts a Prefect flow locally that then schedules a Modal job; the actual work won’t be tracked as part of the same flow run. Tasks - Never decorate tasks with Modal. Use only
@task
— they’ll run inside the Modal container that’s executing your flow. App vs Stub - Newer Modal:
app = <http://modal.App|modal.App>(...)
+
@app.function
. - Older Modal:
stub = modal.Stub(...)
+
@stub.function
. - The same guidance applies to both. Remember - Make sure the Modal runtime has Prefect creds: -
PREFECT_API_URL=<https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>>
-
PREFECT_API_KEY=<YOUR_KEY>
Docs - Run with Modal: Use Prefect with Modal - Connect to Cloud: Connect to Prefect Cloud - Custom names/tags: Custom metadata (names, tags) If you share which Modal API (App vs Stub) you’re using, I can tailor the snippet exactly to your setup.