<@ULVA73B9P> can I set the logging configuration o...
# ask-marvin
g
@Marvin can I set the logging configuration on my workpool?
m
thought for 352.2 seconds
Short answer: there isn’t a special “logging config” switch on a work pool, but you can make every run launched by that pool use your logging settings by putting the right environment variables (or a logging config file) into the work pool’s base job template. Workers and flow-run processes will then pick them up. Common ways to do it - Provide a logging config file and point Prefect to it - Place a
logging.yml
at
PREFECT_HOME/logging.yml
in the runtime environment, or point to it with a setting (see docs below). - Example logging.yml (dictConfig):
Copy code
version: 1
    disable_existing_loggers: false
    formatters:
      standard:
        format: "%(asctime)s | %(levelname)s | %(name)s - %(message)s"
    handlers:
      console:
        class: logging.StreamHandler
        formatter: standard
        level: INFO
        stream: <ext://sys.stdout>
    loggers:
      prefect:
        level: INFO
        handlers: [console]
        propagate: false
    root:
      level: INFO
      handlers: [console]
- Set Prefect logging settings via environment variables - You can set things like: - turning on log forwarding to the API - enabling
log_prints
- adding
extra_loggers
- colors/markup, etc. - The exact setting names are documented in the Settings reference (linked below). Add them to the work pool’s
env
so every flow run from that pool inherits them. How to apply this to a work pool - Edit the work pool’s base job template (UI or CLI) and add your env vars under
variables.env
. - CLI example (Process worker shown, but works similarly for Docker/Kubernetes):
Copy code
prefect work-pool get-default-base-job-template --type process > process.json

  # Edit process.json and add your env vars, for example:
  # {
  #   "job_configuration": { ... },
  #   "variables": {
  #     "env": {
  #       "PREFECT_LOGGING_LOG_PRINTS": "true",
  #       "PREFECT_LOGGING_EXTRA_LOGGERS": "urllib3,boto3",
  #       "PREFECT_LOGGING_SETTINGS_PATH": "/etc/prefect/logging.yml"
  #     }
  #   }
  # }

  prefect work-pool update my-pool --base-job-template process.json
- For deployments, you can also set env per-deployment via
job_variables.env
in your `prefect.yaml`:
Copy code
deployments:
    - name: etl
      work_pool:
        name: my-pool
      job_variables:
        env:
          PREFECT_LOGGING_SETTINGS_PATH: /etc/prefect/logging.yml
          PREFECT_LOGGING_LOG_PRINTS: "true"
Notes - If you also want to change the worker process’s own logs (not just flow runs), set the same env vars in the environment that starts the worker (e.g., systemd service, container env for the worker). - Kubernetes/Docker workers will pass
variables.env
into the pod/container; Process workers set them directly in the subprocess that runs the flow. Docs and references - Logging customization (how to use a logging.yml and available knobs): Logging customization - Settings and profiles (env vars and profiles): Settings and profiles - Managing work pools and job templates: Manage work pools - Customize job variables for deployments: Customize job variables Quick check: what worker type are you using (Process, Docker, or Kubernetes), and do you primarily want to change level/format, forward logs, or include additional libraries’ logs? I can share a tailored template snippet.