<@ULVA73B9P> I have a simple prefect workflow with...
# ask-marvin
a
@Marvin I have a simple prefect workflow with get_run_logger() and a few logger.info() in my flow and task. It works locally, but when I run it in the deployment (so in a container), the logs are not working
m
thought for 169.6 seconds
Thanks for the details — let’s get your logs showing up. Assuming Prefect 3.x:
get_run_logger()
logs are sent to the Prefect API by a background handler; locally they’ll appear immediately in your terminal, but in a deployment (Docker/K8s) a few config details can prevent them from reaching the UI. A few quick clarifying questions to pinpoint this: - Are you on Prefect Cloud or self-hosted Prefect Server? - Which work pool/infrastructure are you using (Docker, Kubernetes, Process)? - Do you see the flow run in the UI with zero log lines, or are you also missing logs in the container stdout? - Any custom logging config (e.g.,
PREFECT_LOGGING_SETTINGS_PATH
or a custom logging.yml) or environment variables set in the image? Most common fixes when logs work locally but not in a containerized deployment: - API configuration must be present in the flow run environment: -
PREFECT_API_URL
and
PREFECT_API_KEY
must be set. If a Prefect worker starts your container, it injects these automatically. If you run the image manually, you must set them yourself. - Ensure logging-to-API is enabled (it is by default): -
PREFECT_LOGGING_TO_API_ENABLED=true
- Verify log levels are not filtering out info logs: -
PREFECT_LOGGING_LEVEL=INFO
(or set per-logger) -
PREFECT_LOGGING_LOGGERS_PREFECT_FLOW_RUNS_LEVEL=INFO
-
PREFECT_LOGGING_LOGGERS_PREFECT_TASK_RUNS_LEVEL=INFO
- If you’re relying on print statements, enable print-to-log: - Decorate the flow with
@flow(log_prints=True)
or set
PREFECT_LOGGING_LOG_PRINTS=true
- Give logs a couple seconds to arrive (they’re batched). You can force-flush in a test: - Call
await APILogHandler.aflush()
at the end of the run - If you ship a custom logging.yml, make sure the Prefect flow/task loggers have the API handler: -
loggers.prefect.flow_runs.handlers: [api]
-
loggers.prefect.task_runs.handlers: [api]
- On K8s, ensure the pod has a terminationGracePeriod so the handler can flush on shutdown. A tiny diagnostic flow you can run in the containerized deployment (or add temporarily) to verify everything:
Copy code
from prefect import flow, task
from prefect.logging import get_run_logger
from prefect.logging.handlers import APILogHandler
from prefect.settings import PREFECT_API_URL, PREFECT_API_KEY, PREFECT_LOGGING_LEVEL

@task
def t():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Task log OK")

@flow(log_prints=True)
def debug():
    logger = get_run_logger()
    print("print -> log if log_prints=True")
    <http://logger.info|logger.info>("Flow log OK")
    t()
    # Force-flush logs to the API (bypasses the batching delay)
    import asyncio
    asyncio.run(APILogHandler.aflush())
    # Quick visibility into config
    print(f"API URL: {PREFECT_API_URL.value()}")
    print(f"Has API key: {bool(PREFECT_API_KEY.value())}")
    print(f"Logging level: {PREFECT_LOGGING_LEVEL.value()}")

if __name__ == "__main__":
    debug()
If logs still don’t show in the UI, enable handler diagnostics: - Set
PREFECT_LOGGING_INTERNAL_LEVEL=DEBUG
in the container/worker — any errors sending logs to the API will be printed to stderr and help pinpoint auth/network/config issues. Helpful docs: - Add logging to flows and tasks - Advanced logging customization (handlers, levels, formats) - Settings and environment variables If you can share: - The deployment’s work pool type - Whether the run shows in the UI with zero logs - Any env vars you’ve set in the image/deployment I can tailor exact steps for your setup.