How can I stream logs to a custom `prefect.log` fi...
# prefect-getting-started
l
How can I stream logs to a custom
prefect.log
file. The logging docu is not covering this. If possible, I would like to set this config in the Python code and not in
~/.prefect/logging.yml
. See here. Thanks.
👍 1
Copy code
import logging
from logging.handlers import RotatingFileHandler
from time import sleep

from prefect import flow, task
from prefect.logging import get_run_logger

# Set up a custom logger
logger = logging.getLogger("prefect")

# Create a handler for writing to a file
handler = RotatingFileHandler(
    "all_logs.log",
    maxBytes=10 * 1024 * 1024,
    backupCount=5,
)
handler.setLevel(logging.DEBUG)

# Add the handler to the logger
logger.addHandler(handler)


# Define the task
@task
def say_hello(name: str) -> None:
    """Say Hello."""
    logger = get_run_logger()
    logger.debug("This is a debug message in the say_hello task.")
    <http://logger.info|logger.info>("We are in the say_hello task.")
    logger.error("This is an error message in the say_hello task.")
    print(f"Hello, {name}!")
    sleep(1)
To answer my own question,
RotatingFileHandler
does the trick. But setting the log level has no effect.
b
Hi Lars!
Just following up after our conversation
For
get_logger
, you could: instantiate it at the top of your script, like so:
Copy code
from prefect import flow, task
from prefect.logging.loggers import get_logger

logger = get_logger("hey-there")

@task
def work(x):
   logger.warning(x)

@flow
def collection_of_work():
    <http://logger.info|logger.info>("Starting work...")
    work.map(range(5))
    
collection_of_work()
How to Stream Prefect Logs to a File <- will be a great reference for you
l
@Bianca Hoch Very helpful. I now have an example of a single logger object streaming to a single log file running.