https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> whats the recommended way to add pref...
# ask-marvin
e
@Marvin whats the recommended way to add prefect logging to tasks across various python scripts?
m
✅ thought for 42.0 seconds
Based on the research, I'll provide you with the recommended approach for adding Prefect logging across your Python scripts. Here's the best practice: 1. Use 
get_run_logger()
 Here's how to implement it in your tasks:
Copy code
python
from prefect import task, flow, get_run_logger

@task
def my_task():
    logger = get_run_logger()
    <http://logger.info|logger.info>("This is a task log message")
    logger.debug("Detailed debug information")
    logger.warning("Warning message")
    logger.error("Error message")

@flow
def my_flow():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Starting the flow")
    my_task()
2. For Multiple Scripts Create a consistent pattern across your scripts:
Copy code
python
# script1.py
from prefect import task, get_run_logger

@task
def task_in_script1():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Executing task in script1")

# script2.py
from prefect import task, get_run_logger

@task
def task_in_script2():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Executing task in script2")
Key Benefits: - Logs are automatically associated with the correct task/flow run - Logs appear in the Prefect UI with proper context - Supports standard logging levels (DEBUG, INFO, WARNING, ERROR, CRITICAL) - Maintains consistency across your entire workflow Additional Tips: 1. You can customize logging behavior through environment variables:
Copy code
bash
# Add logging from external libraries
export PREFECT_LOGGING_EXTRA_LOGGERS="['my_library']"

# Enable Rich markup in logs
export PREFECT_LOGGING_MARKUP=true
2. When using Rich markup (if enabled):
Copy code
python
@task
def formatted_task():
    logger = get_run_logger()
    <http://logger.info|logger.info>("[blue]Processing[/blue] data")
    logger.warning("[yellow]Warning:[/yellow] Check this out")
For more detailed information about customizing logging behavior, you can refer to the Prefect logging documentation. Remember that 
get_run_logger()
should be called within a task or flow context - it won't work outside of these contexts because it needs the Prefect runtime context to properly associate logs with the correct run. Would you like me to elaborate on any particular aspect of logging in Prefect, such as configuration options or advanced logging patterns?
n
id say it depends how much control you need the simplest, least invasive way is to just set 
PREFECT_LOGGING_LOG_PRINTS
to true and all 
print
statements will be logged
but you can also import 
get_run_logger
if you want a python logger to use within runs
e
i'd like to use a python logger instead of prints, so I'd just import get_run_logger() in every script?
n
yea if you need to send those logs to the prefect API
🙌 1
in some cases marvin's suggestion makes sense too
export PREFECT_LOGGING_EXTRA_LOGGERS="['my_library']"
when maybe you have 
my_library
with its own logger, you could set this env var and those would also be posted to the API
Open in Slack
PreviousNext
Powered by