Hi guys, My problem is following: I’d like to pass the logs (standard python logger) from the submodules to prefect logger ( so it can be displayed on the server). I understood, that it can be done with

run config

in flow (I do the local run with the local agent):

run_config=UniversalRun(env={
    "PREFECT__LOGGING__EXTRA_LOGGERS": '["lib1", "lib2"]',
    "PREFECT__LOGGING__LEVEL": "DEBUG",
})

I can ONLY observe

info

level from my libs even if I try to switch on

debug

level! (However the prefect logger shows the information related to

CloudFlowRunner

from

debug

level without any problem). What do I wrong?

I'm having a similar problem trying to overwrite the log format

_env_={'PREFECT__LOGGING__FORMAT':'[%(asctime)s-%(levelname)s - %(name)s]-[%(filename)s:%(funcName)s]-Line %(lineno)d: %(message)s'}

, so mark me as also interested in this question. I've also tried it in the config.toml file to no avail.

This looks right to me. INFO level logs print fine for

lib1

? I’ll take a look at the source, but I can’t think of any reason off the top of my head. Will try that too @Tom Shaffner

INFO

level prints right for both libs without any problem 👍

Info is the default though, right? So if your setting is failing to overwrite you'd only see the issue when you picked something else.

I think he used DEBUG and got DEBUG Prefect logs but INFO logs for the

lib1

and

lib2

Yes, you are right @Kevin Kho that is my problem!

So the source looks right to me. It just attaches the level from the config. I will make a test

Maybe it changes something, but my flow (indeed is a subflow), for which I specify

run_config

is run by the parent flow.

Ok so I tried this out: fake_lib.py

import logging


def function_that_logs():
    logger = logging.getLogger("foo.bar")
    <http://logger.info|logger.info>("INFO from fake_lib!")
    logger.debug("DEBUG from fake_lib!")

theflow.py

from fake_lib import function_that_logs
from prefect import Flow, task
from prefect.run_configs import LocalRun


@task
def foo():
    function_that_logs()


with Flow("extra-loggers-example") as flow:
    foo()

flow.run_config = LocalRun(env={"PREFECT__LOGGING__EXTRA_LOGGERS":'["foo"]',
                                "PREFECT__LOGGING__LEVEL": "DEBUG"})

flow.register("bristech")

I should specify the same run_config for parent and subflow?

The subflow is a new process and subflow logs don’t make it to the parent flow so I would say you need the EXTRA_LOGGERS in any flow that uses that library

Hm, It seems to me that the logs which are outside the run() method cannot be passed. In my case, we have a class which inherits the Task, so we overwrite run() to be compatible. But in the class there are also several functions in which I use logs. Those functions is called in run method and the logs from those functions I want to be able to observe. Do I smth wrong?

I think this should work. I can modify the example a bit. Could you give me a small example so I get a better picture?

@Kevin Kho, to your question above, yes, I was hoping to be able to see the filenames in particular in the UI; I take it the formatting it is doing prevents that? In which case maybe I need to check the logs themselves when I need this?

I need to ask around for this. Will get back to you when I get a response

@Kevin Kho It works now, I restarted my PC, and now debug level works 🤷‍♂️ Thanks for your help!