Felix Schran

01/25/2021, 1:59 PM
Hi all, in prefect core how can I save all the aumatic logging messages to a file? First (before I run the flow), I create a logging file by calling loguru.logger via logger.add("logs.txt")."Some info message before the flow is run") then writes "Some info message before the flow is run" to the file build.txt. Next, I run the flow and then get all these nice logging messages about the tasks. For instance: [2021-01-25 12:30] INFO - prefect.TaskRunner | Task 'xyz': Finished task run for task with final state: 'Success'... How can I add these logging messages to the logging file "logs.txt" from above? So ideally my build.txt file should include in the end: ... | Some info message before the flow is run. ... | Task 'xyz': Finished task run for task with final state: 'Success'...


01/25/2021, 3:59 PM
Hi @Felix Schran! I’d like to learn a little more about your objective. First, are these runs orchestrated with Prefect Cloud/Server? Or are these run using only Prefect Core? You can add all the logs you want! For more on Prefect Loggers, check out:


01/25/2021, 4:00 PM
I’ve answered this before and have a basic example — the key here is
Copy code
import os
from logging import FileHandler

import prefect
from import Local
from prefect import task, Flow

LOG_PATH = os.path.expanduser("~/flow-logs.txt")

def get_logger():
    logger = prefect.context.get("logger")
    return logger

def report_start_day():
    logger = get_logger()

with Flow("flow-that-logs", storage=Local()) as flow:

💯 1

Felix Schran

01/25/2021, 5:38 PM
I am only using prefect core without the prefect cloud backend. Thanks for the example @Zanie When I run that example, I get the output printed: [2021-01-25 173558+0000] INFO - prefect.FlowRunner | Beginning Flow run for 'flow-that-logs' [2021-01-25 173558+0000] INFO - prefect.TaskRunner | Task 'report_start_day': Starting task run... [2021-01-25 173558+0000] INFO - prefect.report_start_day | 2021-01-25 [2021-01-25 173558+0000] INFO - prefect.report_start_day | 4991f380-b29f-4f59-a1a1-295f98fd1c18 [2021-01-25 173558+0000] INFO - prefect.TaskRunner | Task 'report_start_day': Finished task run for task with final state: 'Success' [2021-01-25 173558+0000] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded I would like to exactly write that output into the log file "flow-logs.txt". How can I do that?


01/26/2021, 4:42 PM
If you’re only using core without the backend then you can just add the file handler at the global scope (before calling rather than in a function.
Python supports full customization of handler for existing loggers. Please see