@Marvin is there any way to use the prefect logger in state change handlers?

Yes, you can use the Prefect logger within state change handlers. To access the Prefect logger, you can import

get_run_logger

from

prefect

. You can send messages to the logger in both flows and tasks, including within state change handlers by retrieving a logger instance with

get_run_logger()

, and then calling the standard Python logging methods. Here's a brief example of how you might use

get_run_logger

in your code:

python
from prefect import flow, get_run_logger

@flow(name="log-example-flow")
def logger_flow():
    logger = get_run_logger()
    <http://logger.info|logger.info>("INFO level log message.")

For more information about customization and usage of the Prefect logger, you can refer to the Prefect Logging Guide.

hmmm @Marvin I was having trouble using

get_run_logger

in a state change handler. it was giving me an error saying failed to retrieve logger for current run

It seems you're encountering an issue similar to what has been reported when using

get_run_logger

under specific conditions, resulting in an error citing a failure to retrieve the logger for the current run. From the issues gathered, one particular case that stands out (though it might not be directly related to state handlers) is with using

get_run_logger

in conjunction with DaskTaskRunner and Task.map (issue #9322). The error manifests as logs showing

Flow run '<unknown>'

instead of the expected flow run name. While the exact issue you're describing isn't explicitly reported, these issues suggest that the context required by

get_run_logger

to retrieve the correct logger may not always be reliably present in all execution scenarios. One workaround might be to try and directly use the standard Python logging configuration to create a logger manually, ensuring it isn't dependent on Prefect's context-sensitive

get_run_logger

function. However, if this issue persists and is affecting your use case severely, I would recommend raising a new issue on the Prefect GitHub repository with the specifics of your implementation and the exact error message you are seeing. This could help Prefect's team to identify if there is an underlying issue that needs to be addressed.

@Marvin my error is

prefect.exceptions.MissingContextError: There is no active flow or task run context.

The

MissingContextError

stating "There is no active flow or task run context" occurs when the Prefect logging context is not available or recognized, particularly when attempting to use

get_run_logger()

outside of the expected Prefect flow or task execution. An issue identical to your situation has been discussed in Prefect's GitHub repository (issue #8568). This indicates that calling

get_run_logger()

outside of an active flow or task run context will result in this error. To avoid the error, ensure that you are using

get_run_logger()

within an active flow or task. If the context should be active, and you're still receiving this error, the issue may be a bug or a limitation of the current context provisioning in Prefect. To access runtime context directly, you can use:

python
from prefect.context import get_run_context

However, this function will raise an exception if no runtime context is available. Alternatively, you might try using

FlowRunContext.get()

or

TaskRunContext.get()

that will return

None

instead of throwing an error if the context is not available (runtime context documentation). Be aware that hook functions, such as state change hooks, are client-side and hence may not always have access to a flow or task run logger (state change hooks documentation). If you are attempting to call

get_run_logger()

from a hook or from within a testing environment, similar to what is described in the issue, you may need to adjust your approach. For troubleshooting further, it might be most effective to prepare a minimal reproducible example and discuss it with the Prefect community or raise a new issue in the Prefect GitHub repository.

@Marvin is there any way for me to log from state handlers and have it appear in the prefect ui?

To log from state handlers and have it appear in the Prefect UI, you should be able to use the standard logging library in Python, since Prefect's logger is an extension of the standard Python logger. By setting up proper log handlers and levels, you can ensure that logs emitted during flow and task runs are captured and sent to the Prefect UI for monitoring. However, as per the documentation, it's best to use the hook functions provided by Prefect to react to state changes (Flow and Task Run States). These hooks enable you to execute code in response to specific state transitions in your workflows. State change hooks for flows and tasks can be defined using the

on_*

event parameters for

@flow

and

@task

decorators. Here's an example pattern for creating state change hooks:

python
from prefect import flow, task, FlowRun, TaskRun, State

def my_flow_hook(flow: Flow, flow_run: FlowRun, state: State):
    # This is the required signature for a flow run state change hook.
    # Logging here with Python's standard logging module should show up in the Prefect UI.
    # ...your hook code...

# pass hook as a list of callables
@flow(on_completion=[my_flow_hook])
def my_flow():
    # ...flow definition...

def my_task_hook(task: Task, task_run: TaskRun, state: State):
    # This is the required signature for a task run state change hook.
    # Logging here with Python's standard logging module should show up in the Prefect UI.
    # ...your hook code...

# pass hook as a list of callables
@task(on_failure=[my_task_hook])
def my_task():
    # ...task definition...

Prefect captures logs for your flow and task runs by default, so if you configure logging properly within these hooks, the messages should be transmitted to the Prefect UI (Logging in Prefect). If you find that logs from your hooks are not showing up in the Prefect UI, ensure that your logging configuration is set up correctly, and that your Prefect instance has been configured to capture and display logs. If everything is set up correctly and you're still facing issues, it could be worth checking for any related open issues, or opening a new issue in the Prefect GitHub repository to get help from the Prefect team.