Juanes Grimaldos
07/27/2024, 5:23 PMMarvin
07/27/2024, 5:23 PMget_run_logger
method for retrieving a logger instance, then call the standard Python logging methods. Here are more specific examples for both tasks and flows:
Logging in Tasks
python
from prefect import flow, task, get_run_logger
@task(name="log-example-task")
def logger_task():
logger = get_run_logger()
<http://logger.info|logger.info>("INFO level log message from a task.")
@flow(name="log-example-flow")
def logger_flow():
logger_task()
logger_flow()
Reference
Logging in Flows
python
from prefect import flow
@flow(log_prints=True) # send `print` statements as INFO logs
def my_workflow():
print("I can call any python code here, including prefect tasks or flows")
if __name__ == "__main__":
my_workflow()
Code ExampleJuanes Grimaldos
07/27/2024, 5:24 PM@task(name="log-example-task")
def logger_task():
Marvin
07/27/2024, 5:24 PMpython
import logging
# Set up logging configuration
logging.basicConfig(level=<http://logging.INFO|logging.INFO>, format='%(asctime)s - %(levelname)s - %(message)s')
def perform_task():
logger = logging.getLogger(__name__)
<http://logger.info|logger.info>("This is a log from the external script.")
if __name__ == "__main__":
perform_task()
Prefect Task Script
python
import subprocess
from prefect import flow, task, get_run_logger
@task(name="log-example-task")
def logger_task():
logger = get_run_logger()
result = subprocess.run(["python", "external_script.py"], capture_output=True, text=True)
# Capture and log output from the external script
if result.stdout:
<http://logger.info|logger.info>(result.stdout.strip())
if result.stderr:
logger.error(result.stderr.strip())
@flow(name="log-example-flow")
def logger_flow():
logger_task()
logger_flow()
This approach captures standard output and errors from the external script and logs them in Prefect's logging system.
Additional reference on logging can be found here.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by