https://prefect.io logo
Title
t

Tim Galvin

04/27/2023, 6:33 AM
A question for the brain trust. I am interested in using the
on_completion
and
on_failure
hooks made available for tasks and flows. In the past I would put cleanup / success logic as a distinct
task
in my pipeline. These tasks have their own loggers that would report some items. The following is with regards to callbacks supplied to the
on_completion
and
on_failure
hooks of a flow. What is the correct way of using the
logging
module and a
logger
object to emit info/warn messages that are also recorded by the prefect server? A
logger = get_run_logger()
call in one of my
on_completion
callback functions raised a context error, and
print
statements with
log_print=True
don't seem to be captured either. I guess ultimately I am curious about whether these call back functions are extecuted within or outside a flow context.
j

Justin Trautmann

04/27/2023, 7:30 AM
this works for me
def task_completion_hook(task, task_run, state):
    from prefect.logging.loggers import task_run_logger
    logger = task_run_logger(task_run, task)
    logger.warn("TEST")
t

Tim Galvin

04/27/2023, 7:37 AM
Interesting. Does this work for a flow as well? I have not tried to import the logger in that way before. I will give it a whirl.
@Justin Trautmann - you legend. I used the
flow_run_logger
and passed through the
flow_run
and
flow
objects that are passed to the call back and worked like a charm. I was not able to find them referenced in the docs. Would you happen to know where they are described? I am blind (in the literal sense) and may have overlooked them
j

Justin Trautmann

04/27/2023, 8:38 AM
@Tim Galvin not sure if this is intended usage of the flow/task run logger. i just found it in the
get_run_logger
method.
t

Tim Galvin

04/27/2023, 9:09 AM
As in by reading the source of the
get_run_logger
method?