I'm clearly not understanding logging well here is my sample code

~/my_package/tests/test_logging.py

import structlog
import logging
from prefect.logging import get_run_logger
from prefect import flow


test_logger = logging.getLogger(__name__)

def test():
    structlog.get_logger().info("test from structlog", y=1)
    test_logger.info("test from logging")


@flow()
def flow_func():
    test()


if __name__ == "__main__":
    flow_func.serve(
        name="test",
    )

I have set PREFECT_LOGGING_EXTRA_LOGGERS='my_package' When running from a console this I get these logs

13:36:16.218 | INFO    | prefect.flow_runs.runner - Runner 'test' submitting flow run '8b1f6602-d38b-43b0-964e-29c91a279859'
13:36:16.515 | INFO    | prefect.flow_runs.runner - Opening process...
13:36:16.524 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '8b1f6602-d38b-43b0-964e-29c91a279859'
13:36:18.133 | INFO    | Flow run 'pompous-tench' - Downloading flow code from storage at '.'
2024-12-30 13:36:18 [info     ] test from structlog            y=1
13:36:18.419 | INFO    | Flow run 'pompous-tench' - Finished in state Completed()
13:36:18.734 | INFO    | prefect.flow_runs.runner - Process for flow run 'pompous-tench' exited cleanly.

and from the local UI I see none of the logs. Questions: 1. why is only the structlog showing on the console 2. why are none of the logs being propagated to the UI? (prefect 2.20.10)

hi @tj bollerman you might want to write your own

logging.yml

if you have many different loggers you want to use and be collected by prefect

thanks @Nate - im more trying to understand how they all work. I'd definitely like to use a single logger - we use structlogs a lot and often run functions that might be used inside of flows or outside. So ideally we would just be able to use structlogs everywhere and have the logs show up in the UI

the

logging.yaml

file is where things fit together i suspect you'd want to add something like

# under the loggers section
  my_package:
    level: INFO
    handlers: [console, api]
    propagate: true

thanks ill experiment a bit!!

sounds good! if something is not working as you expect feel free to open a docs issue / PR to clarify and we can add some color

reporting back: so the issue with structlogs (afiak) is it doesnt really come with a name that i can add to the yaml file so that's why it wasnt forwarding. The standard python logging works well tho (i was dumb and had the log level too low before). my biggest piece of feedback is at our company we have a lot of flows in many repos, so configuring all the logging yamls is really burdensome. I dont really see why the local logs using the standard logging library arent passed to prefect by default.

PREFECT_LOGGING_EXTRA_LOGGERS

is a really nice functionality for outside libraries tho