I have a question about logging with async flows in Prefect 2. In short, if we start a flow like this:
Copy code
from prefect import flow, get_run_logger
@flow()
async def my_flow():
logger = get_run_logger()
# Start async process and pass logger
Will the logger also be async?
As a bit of background, we have an async process that spawns multiple other async processes and will create a good deal of logging that we need to catch with the prefect logger. We want to make sure that the logging won't cause any blocking issues. Is passing a logger in this way a safe way to go about it for async code?
z
Zanie
02/09/2023, 12:50 AM
Python doesn’t support async loggers by default
Zanie
02/09/2023, 12:51 AM
The handler that sends logs to our API is always running async in a separate thread, but you’ll never actually await emitting a log message — since it’s just placed on a queue it’s non-blocking.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.