Hi! I'm trying to setup third party (sqlalchemy) logs to output in flow logs. I'm using Docker storage and KubernetesAgent. I do something similar to https://docs.prefect.io/core/concepts/logging.html#extra-loggers in my register_flow.py script and I see some logs when Prefect is capturing flow object for serialization, hence I know it's working. But during the run, no logs from sqlalchemy are written. It seems that logging configuration is not serialized during flow serialization. What is the preferred way to setup third-party logging in flows?

Are you using the env variable? That should work. Normal cases where it doesn’t show up is the logs are handled by different process without Prefect’s handler attached, or you are using a DaskExecutor

No, I'm using logging setup through `getLogger`/`setLevel`

Using the

getLogger

is not preferred. The way to do it would be the env variable below:

export PREFECT__LOGGING__EXTRA_LOGGERS="['snowflake.connector', 'boto3', 'custom_lib']"

but I think the

getLogger

can work if you use a script-based storage by using

stored_as_script=True

with Docker. This stored the Flow as a script, and then evaluates it during execution. Because it’s evaluated, logging configuration will stick. The default serializes the Flow, and the logger settings get lost cuz of this. You can read more abotu script based storage here

Thanks!

I know what you mean though. You can only set one level for all of them with the env var