I'm getting familiar with prefect 2.0b3 and I've r...
# prefect-community
g
I'm getting familiar with prefect 2.0b3 and I've run into a problem with using classes that own a python logger. I get the following error:
ValueError: [TypeError("'_thread.RLock' object is not iterable"), TypeError('vars() argument must have __dict__ attribute')]
Code to reproduce in thread Prefect 1.3 didn't seem to care about loggers being a part of a class used in a flow - any patterns worth adopting to replace this or does this merit a fix?
Copy code
import logging
from prefect import flow, task


class MyModel:
    def __init__(self, name: str):
        self.logger = logging.getLogger(f"model.{name}")
        self.name = name

    def model_things(self, a: int):
        <http://self.logger.info|self.logger.info>(f"Running on value {a}")
        return a > 1


@task
def model_task(model: MyModel):
    outputs = []
    for i in range(10):
        outputs.append(model.model_things(i))
    return outputs


@flow
def model_flow(model):
    model_task(model)


model_flow(MyModel("testing"))
a
you could set extra loggers via environment variable
PREFECT_LOGGING_EXTRA_LOGGERS
- this blog post explains it in detail
k
This looks to be because you are passing the model to the flow as an argument. Not sure what’s going on behind the scenes (something to do with flow parameters maybe?); however, if you re-structure it a bit to create the model in the flow function instead it seems to work:
Copy code
@flow
def model_flow():
    model = MyModel("testing")
    model_task(model)

model_flow()
g
Yeah, there seems to be a problem when serializing the model when passed as a parameter. @Anna Geller I suppose I could pass a
prefect
logger to my objects, but this seems like an extra step that requires classes to accept loggers instead of controlling them independently. I still want that class's logger around for non-prefect testing and experimentation.
k
Just to clarify, you can pass your model with the logger between tasks fine. It’s just passing it to a flow that doesn’t work. Not sure about your use cases; however, I believe flows typically have more typical parameter data types (strings, ints, dicts, etc). So maybe you could create the “MyModel” class in the flow from the simple parameter types and then use that between tasks?
g
Fair point, I'm sure I can get some sort of factory
task
or function implemented. Thanks, Kevin
👍 1