Vincent
01/29/2025, 9:33 PMMarvin
01/29/2025, 9:33 PMget_run_logger()
should provide context-aware logging for both flows and tasks. This means that each logger is supposed to be aware of its respective run context, whether it's a flow or a task. However, if you're only seeing logs from the flow, there might be an issue with how the task logger is being instantiated or used.
Here are a few things to check:
1. Ensure Correct Context: Make sure that get_run_logger()
is being called within the task function, not outside it. Each task should have its own logger call within its function to ensure it logs in the correct context.
2. Inspect Task Configuration: Verify that the task is configured and executed correctly, as misconfigurations might prevent it from logging as expected.
3. Check Execution Environment: Make sure that your environment is set up correctly, especially if you're using Prefect Cloud or a local server. Ensure that your PREFECT_API_URL
is correctly configured if applicable.
Here are some resources that might help:
- Prefect Logging Configuration for more on how logging should work.
- Prefect GitHub Examples for practical examples and setup instructions.
If these do not resolve the issue, consider checking for any open issues or discussions in the Prefect community or GitHub repository.Bianca Hoch
01/29/2025, 10:19 PMfrom prefect import flow, task
from prefect.logging import get_run_logger
@task(name="example-task")
def example_task():
logger = get_run_logger()
<http://logger.info|logger.info>("This is an INFO level log from the task.")
@flow(name="example-flow")
def example_flow():
logger = get_run_logger()
<http://logger.info|logger.info>("This is an INFO level log from the flow.")
example_task()
if __name__ == "__main__":
example_flow()
Vincent
01/29/2025, 10:54 PM11:44:36.861 | INFO | Flow run 'colossal-whippet' - This is an INFO level log from the flow.
11:44:36.883 | INFO | Task run 'example-task-f05' - This is an INFO level log from the task.
when running your code snippet. Thanks!
I omitted an important piece of information, I'm looking at the pytest
logs from a test of my flow run with prefect_test_harness().
To create a minimally reproducible example that mimics my test code:
import pytest
from prefect import flow, task
from prefect.logging import get_run_logger
from prefect.testing.utilities import prefect_test_harness
@task(name="example-task")
def example_task():
logger = get_run_logger()
logger.info("This is an INFO level log from the task.")
@flow(name="example-flow")
def example_flow():
logger = get_run_logger()
logger.info("This is an INFO level log from the flow.")
example_task()
def test_example_flow():
with prefect_test_harness():
flow_state = example_flow(return_state=True)
assert flow_state.is_successful()
if __name__ == "__main__":
example_flow()
the pytest
logs do show both the flow and task logs as intended.
INFO prefect.flow_runs:test_flowtest.py:17 This is an INFO level log from the flow.
INFO prefect.task_runs:test_flowtest.py:11 This is an INFO level log from the task.
So I'll need to continue digging into my other code to see what's going wrong, on the surface it looks the same but thanks for this minimally reproducible logging example!