<@ULVA73B9P> What's the difference between PREFECT...
# ask-marvin
e
@Marvin What's the difference between PREFECT_LOGGING_LEVEL and PREFECT_LOGGING_SERVER_LEVEL?
m
thought for 149.3 seconds
Short answer (Prefect 3.x): -
PREFECT_LOGGING_LEVEL
- What it controls: Verbosity of logs emitted by your client-side processes (flows, tasks, agents, workers, infrastructure). - Default:
INFO
- Applies when: Running your code anywhere (local, worker containers, etc.). -
PREFECT_SERVER_LOGGING_LEVEL
(aka
PREFECT_LOGGING_SERVER_LEVEL
) - What it controls: Verbosity of the Prefect API server’s own logs (e.g., uvicorn/FastAPI) when you’re running a self-hosted Prefect server. - Default:
WARNING
- Applies when: You run the Prefect server yourself (does not affect Prefect Cloud). Notes: - These settings change what you see in logs/console at those processes; they do not control database log retention. - Log retention is handled separately by the server’s vacuum/retention settings, not by logging levels.
e
@Marvin and how do I control the logs coming from my flow's code itself?
m
thought for 122.0 seconds
Great question. In Prefect 3.x, you control logs from your flow/task code (i.e., what you emit via
get_run_logger()
or standard logging) with these levers: 1) Verbosity (what levels are emitted) - Global for all Prefect-run code: -
PREFECT_LOGGING_LEVEL
(default
INFO
) - Example:
Copy code
export PREFECT_LOGGING_LEVEL=DEBUG
- Separate levels just for flow and task run loggers: -
PREFECT_LOGGING_LOGGERS_PREFECT_FLOW_RUNS_LEVEL
-
PREFECT_LOGGING_LOGGERS_PREFECT_TASK_RUNS_LEVEL
- Example:
Copy code
export PREFECT_LOGGING_LOGGERS_PREFECT_FLOW_RUNS_LEVEL=WARNING
    export PREFECT_LOGGING_LOGGERS_PREFECT_TASK_RUNS_LEVEL=DEBUG
2) Where your logs go (API vs console) - Send (or not) to the Prefect API: -
PREFECT_LOGGING_TO_API_ENABLED
(default
True
) - Example:
Copy code
export PREFECT_LOGGING_TO_API_ENABLED=False
- Per-message opt-out from API:
Copy code
logger = get_run_logger()
  <http://logger.info|logger.info>("console only", extra={"send_to_api": False})
- Show flow/task logs in your terminal (by default they go to API only):
Copy code
export PREFECT_LOGGING_LOGGERS_PREFECT_FLOW_RUNS_HANDLERS=api,console
  export PREFECT_LOGGING_LOGGERS_PREFECT_TASK_RUNS_HANDLERS=api,console
3) How to log from your code - Preferred (injects run context automatically and goes to the Prefect log pipeline):
Copy code
from prefect import flow, task, get_run_logger

  @task
  def t():
      logger = get_run_logger()
      logger.debug("debug from task")
      <http://logger.info|logger.info>("info from task")

  @flow
  def f():
      t()
- Capture print statements as logs:
Copy code
@flow(log_prints=True)
  def f():
      print("this becomes a log message")
- Capture your existing Python loggers into Prefect (API) without changing your code:
Copy code
export PREFECT_LOGGING_EXTRA_LOGGERS=myapp,requests,sqlalchemy
Then, logs from
logging.getLogger("myapp")
will be sent to the API and respect
PREFECT_LOGGING_LEVEL
. Notes -
get_run_logger()
logs respect
PREFECT_LOGGING_LEVEL
(they don’t bypass it). -
PREFECT_SERVER_LOGGING_LEVEL
does not affect your flow/task logs; it only affects the Prefect API server process.
e
@Marvin What happens when I just use
structlog
and don't set the PREFECT_LOGGING_EXTRA_LOGGERS var?
m
thought for 34.8 seconds
Short version: nothing special happens. If you use structlog without telling Prefect about those loggers, their output won’t show up in the Prefect run logs (UI/API). You’ll just see them in your process stdout/stderr. Ways to capture structlog logs in Prefect 3.x: - Easiest (use stdlib + add to EXTRA_LOGGERS) 1) Configure structlog to use the standard logging backend. 2) Add your structlog logger names to
PREFECT_LOGGING_EXTRA_LOGGERS
so Prefect attaches its API handler. Example:
Copy code
# structlog setup
  import logging, structlog

  structlog.configure(
      wrapper_class=structlog.stdlib.BoundLogger,
      logger_factory=structlog.stdlib.LoggerFactory(),
      processors=[
          structlog.stdlib.add_log_level,
          structlog.processors.TimeStamper(fmt="iso"),
          structlog.processors.JSONRenderer(),
      ],
  )
  log = structlog.get_logger("myapp")

  # env
  export PREFECT_LOGGING_EXTRA_LOGGERS=myapp
  # optional: control verbosity
  export PREFECT_LOGGING_LEVEL=INFO
Result: structlog→stdlib logger “myapp” gets Prefect’s API handler; messages appear in the run logs and respect
PREFECT_LOGGING_LEVEL
. By default they go to API (not console). Add console if you want:
Copy code
export PREFECT_LOGGING_LOGGERS_PREFECT_EXTRA_HANDLERS=api,console
- Shortcut (no EXTRA_LOGGERS): log to the existing Prefect “extra” logger name
Copy code
log = structlog.get_logger("prefect.extra")
That logger already has the API handler. Trade-off: you’re using a Prefect-owned name. - Programmatic bridge (route structlog events into get_run_logger)
Copy code
from prefect import get_run_logger

  def to_prefect(_, __, event_dict):
      logger = get_run_logger()
      level = (event_dict.get("level") or "info").lower()
      msg = event_dict.get("event") or ""
      # attach remaining struct fields as extras
      extras = {k: v for k, v in event_dict.items() if k not in ("event", "level")}
      getattr(logger, level)(msg, extra=extras)
      raise structlog.DropEvent

  structlog.configure(processors=[to_prefect])
This sends structlog messages directly into Prefect’s flow/task loggers. Notes -
log_prints=True
won’t capture structlog output; it only redirects Python print(). - Without EXTRA_LOGGERS (or the bridge above), Prefect won’t send your structlog messages to the API. - These logs will respect `PREFECT_LOGGING_LEVEL`; you can also add
extra={"send_to_api": False}
if you need to suppress API submission for specific messages (when using stdlib integration).