Hi guys. We're trying to set our log format to log everything JSON so we can consume it in Datadog and search the logs. We'd targetting the following format:

{
  "flow_name": "",
  "flow_id": "",
  "task_name": "",
  "task_id": "",
  "message": "",
  "customer_id": ""
}

The questions we have are: 1. Is using

PREFECT__LOGGING__FORMAT

the correct place to specify the JSON format? 2. How can we handle this additional attribute

customer_id

. Our intent here is for most tasks that are related to our customers to do something like

<http://logger.info|logger.info>('Something', customer_id)

3. To the previous point, how can we extend this further such that we can add arbitrary parameters or objects to the log,

<http://logger.info|logger.info>('Something', my_object,my_second_object)

and have them show up nsted in the json?

I believe you can’t change the logging format for Cloud runs. I think the closest thing you can do is outputting these with the standard Prefect logger. If you have a function that formats the message, you can use it in the task.

def write_log(message):
    res = {"flow_name": prefect.context.flow_name,
     "flow_id": prefect.context.flow_id
     ..
     "message": message
    }
    return res

@task
def some_task():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(write_log(message))

For customer_id, is it’s a parameter, you can get it with

prefect.context.parameters["customer_id"]

. If it’s not a parameter to your flow, then you need to write it when doing

<http://logger.info|logger.info>()

For the arbitrary parameters, you could just accept **kwargs in the write_log function above and format it yourself