The other day I got some advice to look at extra_loggers for handling logs from my packages, which worked great (thanks Anna)! Then I found logging.yml, which was even better.
I had some issues getting Prefect to actually respect logging.yml when I finally deployed everything, and it appears that the problem is specifically with docker infra runs. If I set up an agent for local/process runs it's all good. Is anyone aware of a workaround for this issue, for example, can I put logging.yml somewhere in my docker images to have it get picked up?
I worked around the problem by embedding my logger.yml file in the docker image and setting the env variable (inside the docker image) PREFECT_LOGGING_SETTINGS_PATH to the path of that file. Not sure if I should file a bug report regarding this or if it's expected behavior.
Of course, my work around means that I can no longer have PREFECT_LOGGING_SETTINGS_PATH set in the host vm that actually runs the docker containers, so now I have an opposite problem where the agent-specific logs are ignoring logging.yml instead. That is much less of a problem though.
Final update: By making sure prefect home is properly set, I can ensure that the vm grabs logging.yml from its default location while the docker container is free to use the environment variable. Pretty neat. Still seems like it shouldn't be necessary, but I will hold off on creating an issue unless someone agrees.
08/22/2022, 1:26 PM
@Oscar Björhn, thanks so much for looking into this! I'd love to create a GitHub issue to cross-check with Michael what's our best practice recommendation is to handle that. I agree that it's not clear what exactly should be on the host and what needs to be baked into the image.
If you have any more findings, let's add it to the GitHub issue
@Marvin open "Document whether