Hi all, in prefect core how can I save all the aumatic logging messages to a file? First (before I run the flow), I create a logging file by calling loguru.logger via logger.add("logs.txt"). logger.info("Some info message before the flow is run") then writes "Some info message before the flow is run" to the file build.txt. Next, I run the flow and then get all these nice logging messages about the tasks. For instance: [2021-01-25 12:30] INFO - prefect.TaskRunner | Task 'xyz': Finished task run for task with final state: 'Success'... How can I add these logging messages to the logging file "logs.txt" from above? So ideally my build.txt file should include in the end: ... | Some info message before the flow is run. ... | Task 'xyz': Finished task run for task with final state: 'Success'...

Hi @Felix Schran! I’d like to learn a little more about your objective. First, are these runs orchestrated with Prefect Cloud/Server? Or are these run using only Prefect Core? You can add all the logs you want! For more on Prefect Loggers, check out: https://docs.prefect.io/core/concepts/logging.html https://docs.prefect.io/core/idioms/logging.html

I’ve answered this before and have a basic example — the key here is

addHandler

.

import os
from logging import FileHandler

import prefect
from prefect.environments.storage import Local
from prefect import task, Flow


LOG_PATH = os.path.expanduser("~/flow-logs.txt")


def get_logger():
    logger = prefect.context.get("logger")
    logger.addHandler(FileHandler(LOG_PATH))
    return logger


@task()
def report_start_day():
    logger = get_logger()
    <http://logger.info|logger.info>(prefect.context.today)
    <http://logger.info|logger.info>(prefect.context.flow_run_id)


with Flow("flow-that-logs", storage=Local()) as flow:
    report_start_day()

flow.register(project_name="default")

I am only using prefect core without the prefect cloud backend. Thanks for the example @Zanie When I run that example, I get the output printed: [2021-01-25 173558+0000] INFO - prefect.FlowRunner | Beginning Flow run for 'flow-that-logs' [2021-01-25 173558+0000] INFO - prefect.TaskRunner | Task 'report_start_day': Starting task run... [2021-01-25 173558+0000] INFO - prefect.report_start_day | 2021-01-25 [2021-01-25 173558+0000] INFO - prefect.report_start_day | 4991f380-b29f-4f59-a1a1-295f98fd1c18 [2021-01-25 173558+0000] INFO - prefect.TaskRunner | Task 'report_start_day': Finished task run for task with final state: 'Success' [2021-01-25 173558+0000] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded I would like to exactly write that output into the log file "flow-logs.txt". How can I do that?

If you’re only using core without the backend then you can just add the file handler at the global scope (before calling flow.run()) rather than in a function.