https://prefect.io logo
Title
d

David Ojeda

04/29/2020, 9:20 AM
Hello all! I wanted to introduce a small discussion concerning logging / formatting. Python formatting is one of those things that does not follow its own Python Zen `There should be one-- and preferably only one --obvious way to do it`… We have had
%
-interpolation,
.format
and f-strings. I personally avoid
%
-interpolation;
.format
is very popular but I almost always use f-strings. The
logging
module is the sole exception, in my opinion, where
%
-interpolation should be used, because it uses lazy evaluation:
logger.debug('My object is %s', some_object)
In that example, the
some_object.__str__
method will not be called if the logger level (or is it the handler?) is not low enough to process debug messages. This is different from
logger.debug('My object is %s'%my_object)
or
logger.debug('My object is {}'.format(my_object))
or
logger.debug(f'My object is {my_object}')
because of lazy evaluation: the string will be formatted, then sent to the logger. In a Prefect server setting, where logs are sent to the server, each log record needs to be serializable. This is achieved with
json.dumps
on the
record
received by
prefect.utilities.logging.CloudHandler.emit
This
record
has a field
args
with the args passed to the
logger.log
call, which can have non-serializable objects. What I have seen is that the server logs have many critical messages saying:
Failed to write log with error: TypeError: Object of type X is not JSON serializable
I can change my logging calls to do
logging.debug('My object is %s', str(my_object))
but I really feel like this is a workaround for a bug in
CloudHandler
: perhaps the
args
should not be sent to the server, or unserializable args objects could be redacted. What do you guys think is the appropriate fix here?
j

Jeremiah

04/29/2020, 11:50 AM
Hi @David Ojeda - good news, I think the solution is already PR’d and will be available shortly: https://github.com/PrefectHQ/prefect/pull/2444
👍 1
(Following up a similar issue from @Brett Naul)
d

David Ojeda

04/29/2020, 1:37 PM
Cool! Great news as usual!
😁 2