<@ULVA73B9P> I have hundreds of these entries in t...
# ask-marvin
d
@Marvin I have hundreds of these entries in the prefect 3 worker pod. How do I turn this off?
Copy code
kopf.objects - Handler '_replicate_pod_event' succeeded.
20:51:48.825 | INFO | kopf.objects - Handler '_mark_flow_run_as_crashed' succeeded
m
thought for 395.1 seconds
Those lines come from Kopf (the Kubernetes Operator framework) that Prefect’s Kubernetes observer uses to watch pod events. They’re INFO-level messages like “_replicate_pod_event” and “_mark_flow_run_as_crashed” and are safe to suppress. Two easy ways to quiet them: Option A — Raise the root log level but keep Prefect logs - Set these env vars on your worker pod:
Copy code
PREFECT_LOGGING_ROOT_LEVEL=WARNING
PREFECT_LOGGING_LOGGERS_prefect_LEVEL=INFO
This reduces most third‑party INFO logs while keeping Prefect logs at INFO. Option B — Only suppress Kopf logs - Set the Kopf logger(s) to WARNING:
Copy code
PREFECT_LOGGING_LOGGERS_kopf_LEVEL=WARNING
PREFECT_LOGGING_LOGGERS_kopf.objects_LEVEL=WARNING
If your image/version doesn’t pick up the per-logger env var overrides, use a small logging YAML and point Prefect to it: 1) Create logging.yml:
Copy code
version: 1
disable_existing_loggers: false

loggers:
  kopf:
    level: WARNING
    propagate: true
  kopf.objects:
    level: WARNING
    propagate: true
  prefect:
    level: INFO
    propagate: true

root:
  level: WARNING
  handlers: [console]
2) Mount it into the worker pod (e.g., via a ConfigMap) and set:
Copy code
PREFECT_LOGGING_SETTINGS_PATH=/etc/prefect/logging.yml
Docs: - Prefect 3 logging customization - Kubernetes worker guide Would you prefer to just suppress the logs or disable the Kubernetes observer entirely? If you want to disable it, let me know your Prefect and prefect-kubernetes versions (and the worker image you’re using) and I’ll share the exact setting for your version.
d
@Marvin Both suggestion options Option A — Raise the root log level but keep Prefect logs - Set these env vars on your worker pod:
Copy code
PREFECT_LOGGING_ROOT_LEVEL=WARNING
PREFECT_LOGGING_LOGGERS_prefect_LEVEL=INFO
This reduces most third‑party INFO logs while keeping Prefect logs at INFO. Option B — Only suppress Kopf logs - Set the Kopf logger(s) to WARNING:
Copy code
PREFECT_LOGGING_LOGGERS_kopf_LEVEL=WARNING
PREFECT_LOGGING_LOGGERS_kopf.objects_LEVEL=WARNING
produce still this
kopf.objects
log entries. How can they be suppressed?
m
thought for 877.6 seconds
It looks like your env var names aren’t being applied to the dotted logger name. In Prefect 3.x, nested logging keys use double underscores in env vars, and dotted logger names are split into segments. So: - Use double underscores between nesting keys - Replace dots in logger names with double underscores Examples that will work: - Option A (raise root, keep Prefect at INFO)
Copy code
PREFECT_LOGGING__ROOT__LEVEL=WARNING
PREFECT_LOGGING__LOGGERS__prefect__level=INFO
- Option B (suppress Kopf logs, including the dotted child logger)
Copy code
PREFECT_LOGGING__LOGGERS__kopf__level=WARNING
PREFECT_LOGGING__LOGGERS__kopf__objects__level=WARNING
# If Kopf attaches its own handlers, also stop propagation:
PREFECT_LOGGING__LOGGERS__kopf__propagate=false
PREFECT_LOGGING__LOGGERS__kopf__objects__propagate=false
Tips to validate: - In your worker pod, run:
Copy code
prefect config view --show-sources --show-defaults | grep -i logging
to confirm Prefect is picking up your env vars. - Ensure your env var names don’t contain a literal “.” — Kubernetes env var names and shells won’t accept dots, and Prefect won’t parse them. If you still see
kopf.objects
logs after the above, they may not be coming from the Prefect worker process (e.g., another operator in the cluster is emitting them). In that case, Prefect can’t suppress them; you’d need to filter at your log collector/aggregation layer. For more details and alternative via a logging YAML (with
PREFECT_LOGGING_SETTINGS_PATH
), see Prefect 3 logging customization.