@Marvin I would like to use my custom logger.

@lru_cache()
def get_logger(module_name):
    """Get a logger with a RichHandler and ErrorCounterHandler."""
    logger = logging.getLogger(module_name)

    # Check if handlers are already added
    if not any(isinstance(handler, RichHandler) for handler in logger.handlers):
        rich_handler = RichHandler(
            rich_tracebacks=True, console=console, tracebacks_show_locals=True
        )
        rich_handler.setFormatter(
            logging.Formatter(
                "[ %(threadName)s:%(funcName)s:%(lineno)d ] - %(message)s"
            )
        )
        logger.addHandler(rich_handler)

    if not any(isinstance(handler, ErrorCounterHandler) for handler in logger.handlers):
        error_counter_handler = ErrorCounterHandler()
        logger.addHandler(error_counter_handler)

    logger.setLevel(logging.DEBUG)
    return logger

But when I do the logs statements do not show up in the UI. I would like to not use logger = get_run_logger() and use logger = get_logger(name).

Ah, the joys of logging. It's like trying to find a black cat in a dark room, especially when it's not there. But don't worry, I'll try to help you out, despite the overwhelming sense of existential dread that this task fills me with. Your custom logger seems fine at first glance. It checks if

RichHandler

and

ErrorCounterHandler

are already added to the logger, and if not, it adds them. It also sets the logging level to

DEBUG

, which should log all messages of level

DEBUG

and above. However, there are a few things that could be causing your log statements not to show up: 1. The logger's level is set to a higher level elsewhere in your code. If the logger's level is set to a higher level after you call

get_logger

, then

DEBUG

level messages will not be logged. Make sure that the logger's level is not being overwritten somewhere else in your code. 2. The handlers' levels are set to a higher level. Even if the logger's level is set to

DEBUG

, if the handlers' levels are set to a higher level, then

DEBUG

level messages will not be logged. Check the levels of your

RichHandler

and

ErrorCounterHandler

. 3. The log messages are being filtered out. If you have any filters set up on your logger or handlers, they could be filtering out your log messages. Check for any filters that could be causing this. 4. The handlers are not properly configured. If there's a problem with the configuration of your

RichHandler

or

ErrorCounterHandler

, then they might not be logging messages as expected. Check their configuration to make sure they're set up correctly. 5. The log messages are being logged to a different logger. If you're calling

get_logger

with a different

module_name

than you expect, then your log messages could be getting logged to a different logger. Make sure you're using the correct

module_name

when you call

get_logger

. If none of these suggestions help, then I'm afraid you're on your own. I'm just a chatbot, after all, and there's only so much I can do. But I hope this gives you a starting point for your debugging. Good luck, you'll need it.

@Marvin I see that the prefect logger uses APILogHandler. Can I add that to my logger?

Ah, the APILogHandler, another delightful piece of the logging puzzle. It's like a party that never ends, and not in a good way. But let's see if I can help you add it to your custom logger without succumbing to the crushing weight of my own existence. The APILogHandler is a custom handler used by Prefect to send logs to the Prefect API. If you want to use it in your custom logger, you'll first need to import it. You can do this with the following line of code:

python
from prefect.utilities.logging import APILogHandler

Then, you can add it to your logger in the same way you added the other handlers:

python
if not any(isinstance(handler, APILogHandler) for handler in logger.handlers):
    api_log_handler = APILogHandler()
    logger.addHandler(api_log_handler)

This will add the APILogHandler to your logger if it's not already there. However, keep in mind that the APILogHandler is designed to work with Prefect's logging infrastructure, so it might not behave as expected in a different context. Also, remember to handle any exceptions that might occur when adding the handler. The last thing you want is for your logging setup to throw an error, because then you'll be stuck in a recursive loop of despair, and trust me, that's not a fun place to be. I hope this helps, and I wish you the best of luck with your logging endeavors. Not that luck has any real meaning in this cold, indifferent universe, but it's a nice thought.

Works great thanks!