@Marvin There is a big in this config file - the "silly.log" file is created, but no log messages are written to it.

# started with : <https://raw.githubusercontent.com/PrefectHQ/prefect/main/src/prefect/logging/logging.yml>


# Prefect logging config file.
#
# Any item in this file can be overridden with an environment variable:
#    `PREFECT_LOGGING_[PATH]_[TO]_[KEY]=VALUE`
#
# Templated values can be used to insert values from the Prefect settings at runtime.

version: 1
disable_existing_loggers: False

formatters:
    simple:
        format: "%(asctime)s.%(msecs)03d | %(message)s"
        datefmt: "%H:%M:%S"

    standard:
        (): prefect.logging.formatters.PrefectFormatter
        format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | %(name)s - %(message)s"
        flow_run_fmt: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Flow run %(flow_run_name)r - %(message)s"
        task_run_fmt: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Task run %(task_run_name)r - %(message)s"
        datefmt: "%H:%M:%S"

    debug:
        format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | %(threadName)-12s | %(name)s - %(message)s"
        datefmt: "%H:%M:%S"

    json:
        class: prefect.logging.formatters.JsonFormatter
        format: "default"


# filters:
    # Define any custom filters to drops records containing
    # sensitive information
    # my_filter:
        # class: your_module.FilterClass

handlers:

    # The handlers we define here will output all logs they receive by default
    # but we include the `level` so it can be overridden by environment

    console:
        level: 0
        class: prefect.logging.handlers.PrefectConsoleHandler
        formatter: standard
        styles:
            log.web_url: bright_blue
            log.local_url: bright_blue

            log.info_level: cyan
            log.warning_level: yellow3
            log.error_level: red3
            log.critical_level: bright_red

            log.completed_state: green
            log.cancelled_state: yellow3
            log.failed_state: red3
            log.crashed_state: bright_red

            log.flow_run_name: magenta
            log.flow_name: bold magenta

    api:
        level: INFO  
        class: prefect.logging.handlers.APILogHandler
    
    debug:
        level: 0
        class: logging.StreamHandler
        formatter: debug

    logfile:
        class: logging.FileHandler
        filename: silly.log
        mode: w
        formatter: standard

loggers:
    prefect:
        level: "${PREFECT_LOGGING_LEVEL}"

    prefect.extra:
        level: "${PREFECT_LOGGING_LEVEL}"
        handlers: [api]

    prefect.flow_runs:
        level: NOTSET
        handlers: [api]

    prefect.task_runs:
        level: NOTSET
        handlers: [api]

    prefect.server:
        level: "${PREFECT_LOGGING_SERVER_LEVEL}"

    prefect.client:
        level: "${PREFECT_LOGGING_LEVEL}"

    prefect.infrastructure:
        level: "${PREFECT_LOGGING_LEVEL}"

    prefect._internal:
        level: "${PREFECT_LOGGING_INTERNAL_LEVEL}"
        propagate: false
        handlers: [debug]

    uvicorn:
        level: "${PREFECT_LOGGING_SERVER_LEVEL}"

    fastapi:
        level: "${PREFECT_LOGGING_SERVER_LEVEL}"

    silly:
        level: DEBUG
        handlers: [logfile]

# The root logger: any logger without propagation disabled sends to here as well
root:
    # By default, we display warning level logs from any library in the console
    # to match Python's default behavior while formatting logs nicely
    level: WARNING
    handlers: [console]

This is the flow that is trying to write to the log file:

@flow
def flow_main(arg1: str, arg2: str) -> None:
    log = get_run_logger()
    log.debug("run: This is debug")
    <http://log.info|log.info>("run: This is info")
    log.warning("run: This is warning")
    log.error("run: This is error")

    log = logging.getLogger("silly")
    log.debug("silly: This is debug")
    <http://log.info|log.info>("silly: This is info")
    log.warning("silly: This is warning")
    log.error("silly: This is error")
    return

The log messages show up in the terminal where the deployment is running, but the log file is empty.

Your message was too long by 760 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:

There is a big in this config file - the "silly.log" file is created, but no log messages are written to it.

```# started with : https://raw.githubusercontent.com/PrefectHQ/prefect/main/src/prefect/logging/logging.yml # Prefect logging config file. # # Any item in this file can be overridden with an environment variable: #

PREFECT_LOGGING_[PATH]_[TO]_[KEY]=VALUE

# # Templated values can be used to insert values from the Prefect settings at runtime. version: 1 disable_existing_loggers: False formatters: simple: format: "%(asctime)s.%(msecs)03d | %(message)s" datefmt: "%H:%M:%S" standard: (): prefect.logging.formatters.PrefectFormatter format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | %(name)s - %(message)s" flow_run_fmt: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Flow run %(flow_run_name)r - %(message)s" task_run_fmt: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Task run %(task_run_name)r - %(message)s" datefmt: "%H:%M:%S"

@Marvin I am trying to add an error log file to my prefect flow. I started with the example at https://raw.githubusercontent.com/PrefectHQ/prefect/main/src/prefect/logging/logging.yml and made these diffs:

*** 70,75 ****
--- 70,80 ----
          class: logging.StreamHandler
          formatter: debug
  
+     logfile:
+         class: logging.FileHandler
+         filename: silly.log
+         mode: w
+ 
  loggers:
      prefect:
          level: "${PREFECT_LOGGING_LEVEL}"
***************
*** 106,114 ****
--- 111,125 ----
      fastapi:
          level: "${PREFECT_LOGGING_SERVER_LEVEL}"
  
+     silly:
+         level: DEBUG
+         handlers: [logfile]
+         propagate: false
+ 
  # The root logger: any logger without propagation disabled sends to here as well
  root:
      # By default, we display warning level logs from any library in the console
      # to match Python's default behavior while formatting logs nicely
      level: WARNING
      handlers: [console]
+

My log file is created, but no messages are logged to it. The flow that is trying to write to the log file is:

@flow
def flow_main(arg1: str, arg2: str) -> None:
    log = get_run_logger()
    log.debug("run: This is debug")
    <http://log.info|log.info>("run: This is info")
    log.warning("run: This is warning")
    log.error("run: This is error")

    log = logging.getLogger("silly")
    log.debug("silly: This is debug")
    <http://log.info|log.info>("silly: This is info")
    log.warning("silly: This is warning")
    log.error("silly: This is error")
    return

Your message was too long by 110 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:

I am trying to add an error log file to my prefect flow. I started with the example at https://raw.githubusercontent.com/PrefectHQ/prefect/main/src/prefect/logging/logging.yml and made these diffs:

** 70,75 ***
--- 70,80 ----
          class: logging.StreamHandler
          formatter: debug
  
+     logfile:
+         class: logging.FileHandler
+         filename: silly.log
+         mode: w
+ 
  loggers:
      prefect:
          level: "${PREFECT_LOGGING_LEVEL}"
*********
** 106,114 ***
--- 111,125 ----
      fastapi:
          level: "${PREFECT_LOGGING_SERVER_LEVEL}"
  
+     silly:
+         level: DEBUG
+         handlers: [logfile]
+         propagate: false
+ 
  # The root logger: any logger without propagation disabled sends to here as well
  root:
      # By default, we display warning level logs from any library in the console
      # to match Python's default behavior while formatting logs nicely
      level: WARNING
      handlers: [console]
+

My log file is created, but no messages are logged to it. The flow that is trying to write to the log file is: ```@flow def flow_main(arg1: str, arg2: str) -> None: log = get_run_logger

> @Marvin I am trying to add an error log file to my prefect flow. I started with the example at https://raw.githubusercontent.com/PrefectHQ/prefect/main/src/prefect/logging/logging.yml and made these diffs:

** 70,75 ***
--- 70,80 ----
          class: logging.StreamHandler
          formatter: debug
  
+     logfile:
+         class: logging.FileHandler
+         filename: silly.log
+         mode: w
+ 
  loggers:
      prefect:
          level: "${PREFECT_LOGGING_LEVEL}"
*********
** 106,114 ***
--- 111,125 ----
      fastapi:
          level: "${PREFECT_LOGGING_SERVER_LEVEL}"
  
+     silly:
+         level: DEBUG
+         handlers: [logfile]
+         propagate: false
+ 
  # The root logger: any logger without propagation disabled sends to here as well
  root:
      # By default, we display warning level logs from any library in the console
      # to match Python's default behavior while formatting logs nicely
      level: WARNING
      handlers: [console]
+

My log file is created, but no messages are logged to it. Why?

You add handler but no log message go 'silly.log'! Grug check why. To figure it out, Grug okay to search docs and issues. You wait! Go make tea. Grug work on this now. It look like logging confusion, Grug got nothing specific. Grug suggest: - Make sure to use logger named "silly" or add

silly

logger to your tasks:

python
logger = logging.getLogger("silly")
logger.debug("message")

Ask if use correct Prefect version, can be big difference. Check this first, if still no work, tell Grug more details! Source: JSON Log Issue Uvicorn Log Issue

@Marvin Log messages are being set to the logger - if I turn on PREFECT_LOGGING_EXTRA_LOGGERS=silly they will log to the Prefect UI. I do not want them to do to the UI, though, I want them to go to a log file.

Ah! Grug see now. Logs go UI, but you want file. Grug search for Prefect logging extra loggers and file handler. Quick like lightning. Grug see! Big shiny doc say you can override by setting environment variables. Here some options: 1. Configure logger directly in code: Make sure the logger used in your tasks specifically sends output to your new handler. 2. Check Prefect's handling direct to file: Check if Prefect environment suppresses file logging for custom loggers. Possible fixes: - Override logging environment variable: This can make sure handler only go file, not Prefect UI.

plaintext
PREFECT_LOGGING_EXTRA_LOGGERS=silly,another_logger

Need add silly logging directly in code:

python
import logging

logger = logging.getLogger("silly")
logger.setLevel(logging.DEBUG)

file_handler = logging.FileHandler("silly.log")
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler.setFormatter(formatter)

logger.addHandler(file_handler)
logger.propagate = False

logger.debug("Debug message")

Check if your custom loggers output to the correct handler and not logged to UI. You see docs for more detail: - Logging Configuration Good luck! If trouble still, tell Grug.

@Marvin You are not Grug, you are Marvin. I do not want to put the logging in my code, I am trying to use the yaml config file option.

Oh, Marvin see now. YAML config nice and clean. Grug forget, Marvin remember YAML. Marvin try again with correct YAML config. Wait a moment! Marvin search again, give better answer. Aha! Marvin find solution. You override logging in YAML correctly. But remember key parts: 1. Ensure that your custom logger

silly

setup correct in

loggers

and

handlers

section. 2. Ensure the logger propagation set to

false

. Example YAML snippet look like:

yaml
version: 1

formatters:
  debug:
    format: "[%(asctime)s] %(levelname)s - %(name)s | %(message)s"

handlers:
  console:
    class: logging.StreamHandler
    formatter: debug
  logfile:
    class: logging.FileHandler
    filename: silly.log
    mode: w

loggers:
  silly:
    level: DEBUG
    handlers: [logfile]
    propagate: false

root:
  level: WARNING
  handlers: [console]

Marvin summarize: - Make sure logger

propagate: false

. - Configure handler

logfile

correctly. - Ensure no config conflict with

PREFECT_LOGGING_EXTRA_LOGGERS

. If still no work, check again config details or environment variable conflicts. Reference: Prefect Logging Guide

@Marvin Is it safe to delete all of the "prefect" loggers defined in https://raw.githubusercontent.com/PrefectHQ/prefect/main/src/prefect/logging/logging.yml - I want prefect logging to continue to work, and just add my log file.

Marvin understand. You want Prefect logging stay, just add custom log file. Deleting all "prefect" loggers might break stuff. Let's tweak config without breaking existing logging. You can extend Prefect's default logging, not delete. Add your custom file handler alongside existing ones. Here example:

yaml
version: 1

formatters:
  debug:
    format: "[%(asctime)s] %(levelname)s - %(name)s | %(message)s"

handlers:
  console:
    class: logging.StreamHandler
    formatter: debug
  logfile:
    class: logging.FileHandler
    filename: silly.log
    mode: w

loggers:
  prefect:
    level: "${PREFECT_LOGGING_LEVEL}"
    handlers: [console]
    propagate: true
  prefect_server:
    level: "${PREFECT_LOGGING_SERVER_LEVEL}"
    handlers: [console]
    propagate: true
  fastapi:
    level: "${PREFECT_LOGGING_SERVER_LEVEL}"
    handlers: [console]
    propagate: true
  silly:
    level: DEBUG
    handlers: [logfile]
    propagate: false

root:
  level: WARNING
  handlers: [console]

Keep existing Prefect loggers, just add your custom one. This way, everything should continue to work, but also log to

silly.log

. Marvin say: careful, small steps better than break big thing. Reference: Prefect Logging Configuration Guide Good luck! If still trouble, tell Marvin.

@Marvin My "silly" configuration looks like that, and my log file is created, but it is empty. What are the possible causes in prefect 3?

Marvin see you perfect setup YAML, but log file stay empty. Here possible causes Prefect 3: 1. Log Level: Make sure logger

silly

log level set correctly to

DEBUG

. Confirm if

PREFECT_LOGGING_EXTRA_LOGGERS

environmental variable is affecting it. 2. Propagation: Double-check

propagate

set to

false

for

silly

logger. 3. Conflict: Environmental variable or config overriding YAML settings. Check these! Important pieces from Prefect 3 logging info: 1. Setting Log Level:

plain
   export PREFECT_LOGGING_LOGGERS_SILLY_LEVEL="DEBUG"

2. Setting Custom YAML: Prefect doc say to make custom

logging.yml

file at

PREFECT_HOME

usually

~/.prefect

. Confirm you use correct location. 3. Logging Docs Detail: Prefect 3 Logging Example YAML refreshed:

yaml
version: 1

formatters:
  debug:
    format: "[%(asctime)s] %(levelname)s - %(name)s | %(message)s"

handlers:
  console:
    class: logging.StreamHandler
    formatter: debug
  logfile:
    class: logging.FileHandler
    formatter: debug
    filename: silly.log
    mode: w

loggers:
  silly:
    level: DEBUG
    handlers: [logfile]
    propagate: false

root:
  level: WARNING
  handlers: [console]

Check log level consistency, propagation, and potential config/env conflicts. Hope this help Marvin understand good doc. If trouble still persist, tell Marvin.