https://prefect.io logo
Channels
Powered by
r

Rio McMahon

07/24/2023, 2:58 PM
What is the correct way to capture logging from a 
logger = logging.getLogger(__name__)
object from the 
logging
library? Setting the environmental variable 
PREFECT_LOGGING_EXTRA_LOGGERS=logging
does not seem to work. Thanks.
c

Carlos Velazquez

07/24/2023, 3:39 PM
You can see: 
prefect.get_run_logger()
maybe that help you!
https://docs.prefect.io/2.11.0/concepts/logs/
Copy code
from prefect import flow, get_run_logger

@flow(name="log-example-flow")
def logger_flow():
    logger = get_run_logger()
    <http://logger.info|logger.info>("INFO level log message.")
r

Rio McMahon

07/24/2023, 5:08 PM
Thanks for the response @Carlos Velazquez. Unfortunately this doesn’t work- I have existing code that I am trying to “prefectize”. It uses the 
logging
library and I want to capture that output without modifications to the original code.
c

Carlos Velazquez

07/24/2023, 7:32 PM
@Rio McMahon mmmm in that case, the best way to keep both is, you have to create an Adapter. i have one and works perfectly!
r

Rio McMahon

07/24/2023, 7:36 PM
@Carlos Velazquez can you point me towards documentation/reference code for that?
c

Carlos Velazquez

07/24/2023, 7:43 PM
You can use 
from logging import LoggerAdapter
native from 
logging
Copy code
from logging import LoggerAdapter


class PrefectContextLogging():

    def __init__(self, prefect_logger_adapter: LoggerAdapter) -> None:
        super().__init__(
            name=prefect_logger_adapter.logger.name,
            level=prefect_logger_adapter.logger.level,
            get_logger=lambda x: prefect_logger_adapter.logger
        )
        self._logger_adapter = prefect_logger_adapter
then you can set up the prefect logger in your code: 
PrefectContextLogging(prefect.get_run_logger())
https://docs.python.org/3/library/logging.html#loggeradapter-objects
@Rio McMahon
r

Rio McMahon

07/24/2023, 7:46 PM
Thanks for the detailed response @Carlos Velazquez I will give this a try!
P 1
@Carlos Velazquez is that a valid python snippet? 
class PrefectContextLogging():
doesn’t have a class that it is inheriting from so calling 
super().__init__(...)
doesn’t work unless I am missing something
c

Carlos Velazquez

07/25/2023, 4:58 PM
Hi Rio, yes! it's python and pythonic code! This class it's just called when you required
If you see, it's used the LoggerAdapter
r

Rio McMahon

07/25/2023, 5:00 PM
running
Copy code
from logging import LoggerAdapter
import prefect
from prefect import flow


class PrefectContextLogging():

    def __init__(self, prefect_logger_adapter: LoggerAdapter) -> None:
        super().__init__(
            name=prefect_logger_adapter.logger.name,
            level=prefect_logger_adapter.logger.level,
            get_logger=lambda x: prefect_logger_adapter.logger
        )
        self._logger_adapter = prefect_logger_adapter
        
@flow
def test():
    PrefectContextLogging(prefect.get_run_logger())
test()
returns 
TypeError: object.__init__() takes exactly one argument (the instance to initialize)
c

Carlos Velazquez

07/25/2023, 5:03 PM
So, well Yesterday i shared with you an example code, but you need to put it these inside your code, in that case, you need to make the Adapter for set up this conf in Logging lib!
test this:
Copy code
@flow(name=name_flow_you_want)
async def test_flow() -> None:
    prefect.get_run_logger().info('Running flow test')

    log = PrefectContextLogging(prefect.get_run_logger())

    log.add_context(ctx_foo='bar')
    log.add_data(data_foo='bar')
And also you can make and adapter and just use logging instead 
prefect.get_run_logger()
If you don't used PrefectContextLogging works Logging.logger simply! idk maybe like this:
Copy code
logger = logging.getLogger(__name__)

def set_logget_prefect(logger) -> None:
    global logger
    logger = logger
And inside the flow you just do it this:
Copy code
set_logget_prefect(PrefectContextLogging(prefect.get_run_logger()))

<http://logger.logger.info|logger.logger.info>('info in prefect')
but if you don't set the prefect Adapter you also keep use it this:
Copy code
<http://logger.logger.info|logger.logger.info>('info simply')
that is an Adapter @Rio McMahon
r

Rio McMahon

07/25/2023, 5:12 PM
Awesome thanks for the help @Carlos Velazquez
👀 1
c

Carlos Velazquez

07/25/2023, 5:14 PM
Don't worry! Ping me if these solutions works for you!! @Rio McMahon
Powered by