https://prefect.io logo
Title
a

Adam

10/26/2022, 2:21 PM
Hi team, I’m a bit lost in how to configure JSON logging in Prefect 2.0. I see this concept document and this api reference on the JSONFormatter, but I can’t seem to find how to marry the two together. What we’re trying to achieve is: 1. Prefect Cloud 2.0 still shows logs as is 2. Kubernetes jobs output logs in JSON so that they can be ingested by datadog (we want each log entry to have the flow_name, task_name, etc. to make it easily searchable back in time).
1
k

Khuyen Tran

10/26/2022, 3:42 PM
All logging configurations is in
~/.prefect/logging.yml
. You can change the setting of the logging by typing:
prefect config set PREFECT_LOGGING_[PATH]_[TO]_[KEY]=VALUE
To change the formatter to json, you can type:
prefect config set PREFECT_LOGGING_HANDLERS_CONSOLE_FORMATTER=json

prefect config set PREFECT_LOGGING_HANDLERS_CONSOLE_TASK_RUNS_FORMATTER=json

prefect config set PREFECT_LOGGING_HANDLERS_CONSOLE_FLOW_RUNS_FORMATTER=json
Or simply change the value in
~/.prefect/logging.yml
handlers:

    # The handlers we define here will output all logs they receive by default
    # but we include the `level` so it can be overridden by environment

    console:
        level: 0
        class: logging.StreamHandler
        formatter: json

    console_flow_runs:
        level: 0
        class: logging.StreamHandler
        formatter: json

    console_task_runs:
        level: 0
        class: logging.StreamHandler
        formatter: json

    orion:
        level: 0
        class: prefect.logging.handlers.OrionHandler
a

Adam

10/26/2022, 9:51 PM
Thank you @Khuyen Tran
Would recommend you update those docs too, for the next people with this question 🙂
👍 1
@Khuyen Tran where would I apply such settings in the code itself? We’re using Prefect Cloud 2.0 using KubernetesJob executors. So would I set these as environment variables in the Kubernetes Job block?
k

Khuyen Tran

10/27/2022, 4:48 PM
Yes, you can set the environment variables (
environment
field) to be the above. For example,
{
  "PREFECT_LOGGING_HANDLERS_CONSOLE_FORMATTER": "json",
  "PREFECT_LOGGING_HANDLERS_CONSOLE_TASK_RUNS_FORMATTER": "json",
  "PREFECT_LOGGING_HANDLERS_CONSOLE_FLOW_RUNS_FORMATTER": "json"
}
@terrence I added this to Discourse, but I think it is also good to add this to the documentation in the formatters section. What do you think?
💯 1
t

terrence

10/27/2022, 5:45 PM
Thanks @Khuyen Tran I will add this to the roadmap.
:gratitude-thank-you: 1
a

Adam

10/28/2022, 6:08 PM
Thanks @Khuyen Tran. We’ve enabled that but it appears its not really sending valid JSON. It seems its sending a Python dictionary:
{'name': 'prefect.flow_runs', 'msg': "Finished in state Completed('All states completed.')", 'args': [], 'levelname': 'INFO', 'levelno': 20, 'pathname': '/usr/local/lib/python3.9/site-packages/prefect/engine.py', 'filename': 'engine.py', 'module': 'engine', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 366, 'funcName': 'begin_flow_run', 'created': 1666970625.7505996, 'msecs': 750.5996227264404, 'relativeCreated': 5440.329074859619, 'thread': 140168457979712, 'threadName': 'MainThread', 'processName': 'MainProcess', 'process': 7, 'flow_run_name': 'fancy-alpaca', 'flow_run_id': '61694402-3773-4e93-a24d-55318dbe0de1', 'flow_name': 'my_new_flow', 'send_to_orion': False}
Note the single quotes and the use of types like
False
and
None
Perhaps you’re just
print(some_dict)
instead of
json.dumps(some_dict)
?
@Khuyen Tran https://github.com/PrefectHQ/prefect/blob/orion/src/prefect/logging/formatters.py#L33 Shouldn’t
str
be
json.dumps
instead? A string representation of a dict is not really JSON…
I’ve opened a small PR here: https://github.com/PrefectHQ/prefect/pull/7377
🎉 1
k

Khuyen Tran

10/31/2022, 3:14 PM
great PR! Thanks for submitting one