@Marvin i'd like to persist the logs from each flow run for future access. Is this possible?

Persisting logs from your flow runs in Prefect is indeed possible, and you're in luck because it's quite straightforward in both Prefect 2.x and 3.x versions. For Prefect 2.x, you can configure your flows to log information using the logger provided by Prefect: - Get a Logger: Use

get_run_logger()

to access the logger within your flow or task. Here's a basic example of how you can implement logging in your flow:

python
  from prefect import flow, get_run_logger
  
  @flow
  def my_flow():
      logger = get_run_logger()
      <http://logger.info|logger.info>("This is a log message for my_flow")

- View Logs in UI: Once your log is set up, you can view these logs in the Prefect Cloud UI or your self-hosted Prefect server. More details can be found in the Prefect 2.x documentation. For Prefect 3.x, the process is quite similar: - Use Prefect Logger: Like in 2.x, you can use

get_run_logger()

from the

prefect.logging

module to emit log messages. This allows you to log messages that are aware of the current flow or task run context for more detailed information. - Access Logs: Prefect automatically logs flow and task events, including state changes and errors, which you can access through the Prefect UI, CLI, or API. Need a more comprehensive approach? The 3.x documentation can offer further insights. No existential angst here, just some useful logging insight!

you can fetch them on demand from the CLI if have the UUID https://github.com/prefecthq/prefect/blob/main/src/prefect/cli/flow_run.py#L219-L338 the implementation of CLI would be helpful if you wanted your own logic for doing that

Thanks @Nate, this looks useful.

pretty sure we already use a rich log handler