Hi all. I’m able to stream my program’s logs to the UI setting this ENV variable:

export PREFECT_LOGGING_EXTRA_LOGGERS='my-loggers-name'

Now, when I run my codebase in multiprocessing these logs are not shown in the UI. Anyone has some tips how I still stream the logs of

my-loggers-name

to the UI while using multiprocessing? EDIT: the

root

Logger is also ignored..

Are you able to set the logger as an environment variable in the run config or in the agent start command you are using like in this example? https://discourse.prefect.io/t/vertex-agent-extra-loggers/821/3

I’m on prefect 2.0, don’t think the agent has a

--env

var? Also, I can see already sone differences using

spawn

vs

fork

in multiprocessing, but still no luck to see these in the UI. (single process still works)

ah apologies that was for 1.0. Are you able to set the extra loggers variable within the code like in this example: https://discourse.prefect.io/t/how-to-use-logging-in-prefect-a-tutorial-by-andrew-brookins/197/4#ray-3? Where are you setting that variable using the export statement?

Yes I am. In my prefect code I’m just using the default taks_runner. (No Dask or Ray). But in a task, I call a function that uses multiprocessing. I can see those logs in my console, but not the UI. When I run the same code, without multiprocessing, the logs are visible in the UI. Which means this

PREFECT_LOGGING_EXTRA_LOGGERS

is actually being detected, and the UI also shows the logs from my applications’s logger (or even the root logger). Riddle to solve, why are these logs outputted in my application’s multiprocessing code not visible? UPDATE: When I run my application multiprocessing on

spawn

I don’t see my applications logger in the console. When running on

fork

I see it in the console, but not in the UI

Appreciate the information! I don't believe that the flow/task run context is shared between processes automatically, so you would have to pass a copy context through to each process. We will have our team take a look.