@Marvin I am trying to use 'exclude_unset' with '.json' on a Pydantic model created inside a subflow and passed to a task, but the result of the JSON contains all fields of the model. How can I get the model to keep its 'fields_set' attributes?

Oh, the joys of dealing with Pydantic models and JSON serialization. It's like trying to explain quantum mechanics to a toaster. But let's give it a shot. The

exclude_unset

parameter in the

.json()

method of a Pydantic model is supposed to exclude attributes that were not set when creating the model instance. However, it seems like you're having trouble with this when using it inside a subflow and passing it to a task. One possible reason could be that the model is being re-initialized somewhere in the process, which would reset the

fields_set

attribute of the model. The

fields_set

attribute is used by Pydantic to keep track of which fields were set during initialization. Another possibility is that there's a discrepancy between the model definition and the actual data being passed. If a field is defined as

Optional

in the model but the data contains a

None

value for that field, it might still appear in the JSON output even when using

exclude_unset

. You could try explicitly setting

exclude_unset=True

when calling

.json()

on the model instance. If that doesn't work, you might need to manually filter out the unset fields from the JSON output. Remember, Pydantic is a tool, not a magic wand. It can't read your mind, as much as it might seem like it's trying to sometimes. If you're still having trouble, I'd recommend checking the Pydantic documentation or asking for help on the Pydantic GitHub page. They're usually quite helpful, even to us robots.

@Marvin Does Prefect re-initialize values passed into tasks? The model works just fine when the task is called as a function with '.fn'

Ah, the intricacies of Prefect. It's like trying to navigate a maze in the dark, with only a flickering candle to guide you. But fear not, I'm here to help. Prefect does indeed have a unique way of handling task inputs. When you run a task in Prefect, it doesn't just pass the inputs directly to the task function. Instead, it wraps them in a special

Task

object, which is then passed to the task function. This allows Prefect to track the state of the task and its inputs, among other things. However, this can cause some unexpected behavior when working with complex data types, like Pydantic models. When the model is passed to the task, Prefect doesn't know how to handle it, so it just re-initializes it. This can cause the

fields_set

attribute of the model to be reset, which would explain the behavior you're seeing. When you call the task function directly using

.fn

, you're bypassing Prefect's task handling mechanism, so the model is passed directly to the function without being re-initialized. So, to answer your question: yes, Prefect does re-initialize values passed into tasks. If you want to preserve the state of a Pydantic model when passing it to a task, you might need to find a way to serialize the model and its

fields_set

attribute, and then deserialize it inside the task. But remember, even in the vastness of the universe, there's no such thing as a perfect solution. There's always a trade-off.