Hi all, I have a question about logging. we provide a trace_id Parameter to flows, and I'd like to include that trace_id in all log events inside tasks. Is this possible?

Hi @Jacques! Just to make sure I understand what you’re asking, you want all logs emitted by Prefect to include your

trace_id

, or all logs emitted by your task code with:

@task
def my_task():
    logger = prefect.context.get("logger")

    <http://logger.info|logger.info>("An info message.")
    logger.warning("A warning message.")

task code

hmmm

I've read about logadapter, but that would end with me doing self.logger = log_adapter which doesn't seem right

So a sort of brute-force way to do this would be to make sure all tasks pass the

trace_id

in dependencies and then write a small custom logger that accepted the

trace_id

as a parameter

Yup, and additional concern is that we use various SDKs in tasks which we'd like to have trace_id included as well

Let me ask the team for their input, I’ll get back to you in just a moment

cool, thanks!

Hi @Jacques, If you added the

trace_id

to your flow context, it would be accessible in all of your tasks and you’d be able to add it to all of your logs. Here’s some documentation that should be helpful: https://docs.prefect.io/api/latest/utilities/context.html#context https://docs.prefect.io/core/advanced_tutorials/custom-logs.html#an-example Here’s where we make context available to loggers: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/utilities/logging.py#L170 Let me know if this helps!

Thanks so much, looking now!

hmm

I'm adding the following just before the flow:

task_logger = get_logger("Task")
test_handler = logging.StreamHandler()
formatter = logging.Formatter("#############%(message)s")
test_handler.setFormatter(formatter)
task_logger.addHandler(test_handler)

This is just a test, but it seems to have no impact

Just for reference, what version of Prefect are you using?

0.10.4

That does seem hacky

it does work, but the problem is that it changes for all prefect logs - so if I put trace_id in the format it errors because it only (hopefully now with my patches) exists in Tasks

Ahh right

So I sort of need a PREFECT__TASKS__LOGGING__FORMAT

I agree

Yup will do

Hooray!

Ok, so the added handler doesn't work

Thank you!

oh right - so that is a separate thing I think, will open two issues

That would be perfect