https://prefect.io logo
Join Slack
Powered by
What is the correct way to capture logging from a ...
# ask-community
r
What is the correct way to capture logging from a 
logger = logging.getLogger(__name__)
object from the 
logging
library? Setting the environmental variable 
PREFECT_LOGGING_EXTRA_LOGGERS=logging
does not seem to work. Thanks.
c
You can see: 
prefect.get_run_logger()
maybe that help you!
https://docs.prefect.io/2.11.0/concepts/logs/
Copy code
from prefect import flow, get_run_logger

@flow(name="log-example-flow")
def logger_flow():
    logger = get_run_logger()
    <http://logger.info|logger.info>("INFO level log message.")
r
Thanks for the response @Carlos Velazquez. Unfortunately this doesn’t work- I have existing code that I am trying to “prefectize”. It uses the 
logging
library and I want to capture that output without modifications to the original code.
c
@Rio McMahon mmmm in that case, the best way to keep both is, you have to create an Adapter. i have one and works perfectly!
r
@Carlos Velazquez can you point me towards documentation/reference code for that?
c
You can use 
from logging import LoggerAdapter
native from 
logging
Copy code
from logging import LoggerAdapter


class PrefectContextLogging():

    def __init__(self, prefect_logger_adapter: LoggerAdapter) -> None:
        super().__init__(
            name=prefect_logger_adapter.logger.name,
            level=prefect_logger_adapter.logger.level,
            get_logger=lambda x: prefect_logger_adapter.logger
        )
        self._logger_adapter = prefect_logger_adapter
then you can set up the prefect logger in your code: 
PrefectContextLogging(prefect.get_run_logger())
https://docs.python.org/3/library/logging.html#loggeradapter-objects
@Rio McMahon
r
Thanks for the detailed response @Carlos Velazquez I will give this a try!
P 1
@Carlos Velazquez is that a valid python snippet? 
class PrefectContextLogging():
doesn’t have a class that it is inheriting from so calling 
super().__init__(...)
doesn’t work unless I am missing something
c
Hi Rio, yes! it's python and pythonic code! This class it's just called when you required
If you see, it's used the LoggerAdapter
r
running
Copy code
from logging import LoggerAdapter
import prefect
from prefect import flow


class PrefectContextLogging():

    def __init__(self, prefect_logger_adapter: LoggerAdapter) -> None:
        super().__init__(
            name=prefect_logger_adapter.logger.name,
            level=prefect_logger_adapter.logger.level,
            get_logger=lambda x: prefect_logger_adapter.logger
        )
        self._logger_adapter = prefect_logger_adapter
        
@flow
def test():
    PrefectContextLogging(prefect.get_run_logger())
test()
returns 
TypeError: object.__init__() takes exactly one argument (the instance to initialize)
c
So, well Yesterday i shared with you an example code, but you need to put it these inside your code, in that case, you need to make the Adapter for set up this conf in Logging lib!
test this:
Copy code
@flow(name=name_flow_you_want)
async def test_flow() -> None:
    prefect.get_run_logger().info('Running flow test')

    log = PrefectContextLogging(prefect.get_run_logger())

    log.add_context(ctx_foo='bar')
    log.add_data(data_foo='bar')
And also you can make and adapter and just use logging instead 
prefect.get_run_logger()
If you don't used PrefectContextLogging works Logging.logger simply! idk maybe like this:
Copy code
logger = logging.getLogger(__name__)

def set_logget_prefect(logger) -> None:
    global logger
    logger = logger
And inside the flow you just do it this:
Copy code
set_logget_prefect(PrefectContextLogging(prefect.get_run_logger()))

<http://logger.logger.info|logger.logger.info>('info in prefect')
but if you don't set the prefect Adapter you also keep use it this:
Copy code
<http://logger.logger.info|logger.logger.info>('info simply')
that is an Adapter @Rio McMahon
r
Awesome thanks for the help @Carlos Velazquez
👀 1
c
Don't worry! Ping me if these solutions works for you!! @Rio McMahon
Open in Slack
PreviousNext
Powered by