Logging -- we've added `log_stdout=True` to all of...
# prefect-community
Logging -- we've added
to all of our tasks, and the log messages are showing up just fine in the agent's stdout, but we'd like them to also be available on the web ui. We've discovered that by getting the logger from the context, we can do this, but this is a pretty invasive change for our codebase (most of our modules are using standard logging and we don't want to change that). Is there any magic we can invoke to get our logger messages to show up on the web ui?
Hi Dolor, You're saying that with
print statements from tasks show up as logs locally, but aren't visible in the webui?
not print statements though
Copy code
import logging
_LOGGER = logging.getLogger(__name__)
<http://_LOGGER.info|_LOGGER.info>("I'm a log message")
Oh wait, it sounds like you're expecting
to properly pick up logs generated via other loggers within your tasks? This won't be the case, the way python's logging `StreamHandler`s work our stdout redirection won't properly catch these. You're only seeing things work locally because if you stream local logs through the local agent you're actually just streaming all process output, which is different.
our _LOGGER is by default logging to stdout
ie the above code snippet emits to STDOUT
(not to file or anything)
Yes, but python's logging mechanism grabs
early, meaning that redirecting it in a task won't actually work.
Our stdout redirection works by monkeypatching
, if a library grabs a reference to
before we can do that, the redirection won't work.
You'll need to actually plugin your loggers to prefect's logging mechanism to get what you're asking for.
interesting -- ok, will try that, ty
hrm, I don't have control over all the loggers -- they are in downstream modules/libraries
Python loggers are singletons and are hierarchical. Assuming the downstream libraries don't configure their loggers themselves on import (and only create them) you can make use of prefect's
configuration. Loggers with names specified in this list will be configured to log to prefect. Since loggers are hierarchical (and assuming the libraries are using
for the logger names) this would be a list of package names:
Copy code
# in your ~/.prefect/config.toml
extra_loggers = ['package1', 'package2']
yeah ... that list of packages is a moving target in my enviorrment, so a priori I don't know them
I think I can get them all using
Copy code
loggers = [logging.getLogger(name) for name in logging.root.manager.loggerDict]
though ๐Ÿ™‚
You could also configure via an environment variable, e.g.
PREFECT__LOGGING__EXTRA_LOGGERS="['package1', 'package2', ...]"
if that's easier. Doing it programmatically is fine, but won't transfer to other processes if you're e.g. using a
yeah, sadly I don't have the list of packages (and even if I did, the list is too volatile to maintain). Yeah we're using a dask executor ... I guess we'll do it once for each @task
That might get chaotic. You'd need to ensure whatever modifications you make are idempotent and threadsafe so you don't add the same handlers more than once.
yeah ๐Ÿ˜•
You also don't need to do that for everything in
, just the root parents, e.g.
but not
. Most python libraries I interact with don't use logging, I'm surprised the list you do is so large you can't statically configure it once.
If you have a superset of all libraries you might use, you could add that to the configuration, the library doesn't need to be present to configure its logger.
good idea -- I'll capture just the root of the namespace, as you say
trying it out, thanks again ๐Ÿ™‚
๐Ÿš€ 1
totally worked -- thank you @Jim Crist-Harif
๐Ÿš€ 1
Excellent! Glad to hear it. Let us know if you run into any other issues.
@Marvin archive โ€œI donโ€™t see all the logs that I expect when using log_stdout=Trueโ€