https://prefect.io logo
Powered by
Title
t

Thomas Hoeck

09/03/2020, 1:14 PM
Hi all. How do you handle logging if you want to be able to run a task separatly for e.g. unit-test: 
from prefect import task
import prefect
@task
def extract():
logger = prefect.context.get("logger")
logger.debug("Executing")
return [1,2,3]
extract.run()
as this yields 
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-25-9ca06496a242> in <module>
     11 
     12 
---> 13 extract.run()

<ipython-input-25-9ca06496a242> in extract()
      7 def extract():
      8     logger = prefect.context.get("logger")
----> 9     logger.debug("Executing")
     10     return [1,2,3]
     11 

AttributeError: 'NoneType' object has no attribute 'debug'
d

Dylan

09/03/2020, 2:10 PM
Hi @Thomas Hoeck! In the context of a unit test, I would suggest mocking 
prefect.context
and the 
logger
using 
MagicMock
or the like
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock
t

Thomas Hoeck

09/03/2020, 2:52 PM
What if just want to run it in a Jupyter Notebook doing my development work? Is there a way to establish a context?
@Dylan as in I import the task and run it in a Jupyter Notebook with .run()
d

Dylan

09/03/2020, 2:59 PM
You might be able to 
import prefect
and call the logger directly inside your task
Let me double check
That doesn’t work
The only other thing I can think of that would work without changing task code so that it could run outside of a prefect context would be to implement some sort of 
Logger
to replace the one provided by the prefect context
and then setting the task up like: 
if prefect.context:
  logger = prefect.context.get("logger")
else:
  logger = LocalLogger()
But you’re right, everything else inside a task’s 
run
function should work outside of a prefect context
Hey Thomas, after a discussion with the team I opened an issue for this: https://github.com/PrefectHQ/prefect/issues/3256
Follow this issue for more updates! =]
i

Iain Dillingham

09/04/2020, 9:13 AM
Thanks for the discussion on this. I encountered the same issue but resolved it by importing 
get_logger
from 
prefect.utilities.logging
. I get the logger by calling that function rather than through the context object.
get_logger
is also typed, so mypy doesn't complain.
t

Thomas Hoeck

09/04/2020, 9:31 AM
Nice @Iain Dillingham! But do you get the root logger then or in the context of the task? Still it is mentioned as the suggested way to get logger is https://docs.prefect.io/core/concepts/logging.html#logging-configuration
i

Iain Dillingham

09/04/2020, 9:34 AM
Ah, good point: it returns the root logger by default. And yes, this isn't the documented way of getting the logger.
👍 1
t

Thomas Hoeck

09/04/2020, 9:39 AM
@Dylan will follow along 👀
💯 1
#prefect-communityJoin Slack
Powered by