Moe
08/18/2023, 7:06 PMlogging.yaml
work? E.g. what python logging.handlers
are available to specify? What attributes is it a must for us to specify for the handlers, formatters, loggers etc.? I tried using a simple regular FileHandler seems Prefect doesn't allow it because all my logs on UI disappeared and none written to local.
handlers:
file:
level: 0
class: logging.handlers.FileHandler
filename: C:\...\prefect.log
formatter: standard
Jake Kaplan
08/18/2023, 7:20 PM# Prefect logging config file.
#
# Any item in this file can be overridden with an environment variable:
# `PREFECT_LOGGING_[PATH]_[TO]_[KEY]=VALUE`
#
# Templated values can be used to insert values from the Prefect settings at runtime.
version: 1
disable_existing_loggers: False
formatters:
simple:
format: "%(asctime)s.%(msecs)03d | %(message)s"
datefmt: "%H:%M:%S"
standard:
(): prefect.logging.formatters.PrefectFormatter
format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | %(name)s - %(message)s"
flow_run_fmt: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Flow run %(flow_run_name)r - %(message)s"
task_run_fmt: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Task run %(task_run_name)r - %(message)s"
datefmt: "%H:%M:%S"
debug:
format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | %(threadName)-12s | %(name)s - %(message)s"
datefmt: "%H:%M:%S"
json:
class: prefect.logging.formatters.JsonFormatter
format: "default"
# filters:
# Define any custom filters to drops records containing
# sensitive information
# my_filter:
# class: your_module.FilterClass
handlers:
# The handlers we define here will output all logs they receieve by default
# but we include the `level` so it can be overridden by environment
file:
level: INFO
class: logging.FileHandler
formatter: standard
filename: 'prefect.log'
console:
level: 0
class: prefect.logging.handlers.PrefectConsoleHandler
formatter: standard
styles:
log.web_url: bright_blue
log.local_url: bright_blue
log.info_level: cyan
log.warning_level: yellow3
log.error_level: red3
log.critical_level: bright_red
log.completed_state: green
log.cancelled_state: yellow3
log.failed_state: red3
log.crashed_state: bright_red
log.flow_run_name: magenta
log.flow_name: bold magenta
api:
level: 0
class: prefect.logging.handlers.APILogHandler
debug:
level: 0
class: logging.StreamHandler
formatter: debug
loggers:
prefect:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [file]
prefect.extra:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [api]
prefect.flow_runs:
level: NOTSET
handlers: [api]
prefect.task_runs:
level: NOTSET
handlers: [api]
prefect.server:
level: "${PREFECT_LOGGING_SERVER_LEVEL}"
prefect.client:
level: "${PREFECT_LOGGING_LEVEL}"
prefect.infrastructure:
level: "${PREFECT_LOGGING_LEVEL}"
prefect._internal:
level: "${PREFECT_LOGGING_INTERNAL_LEVEL}"
propagate: false
handlers: [debug]
uvicorn:
level: "${PREFECT_LOGGING_SERVER_LEVEL}"
fastapi:
level: "${PREFECT_LOGGING_SERVER_LEVEL}"
# The root logger: any logger without propagation disabled sends to here as well
root:
# By default, we display warning level logs from any library in the console
# to match Python's default behavior while formatting logs nicely
level: WARNING
handlers: [console]
handlers:
file:
level: INFO
class: logging.FileHandler
formatter: standard
filename: 'prefect.log'
and
loggers:
prefect:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [file] <-- add the handler
Moe
08/18/2023, 7:21 PMlogging.FileHandler
for the regular but others such as timed handler are specified as logging.handlers.TimedRotatingFileHandler