@Marvin can I reliably use the INPUTS caching policy, if a Pydantic model is one of the inputs? It seems to give me inconsistent results.

To provide you with the most accurate answer, I need to confirm the version of Prefect you are using. Could you please let me know if you're using Prefect 2.x, Prefect 3.x, or another version? This will help tailor the solution to the nuances of your Prefect version.

yes, it should work the same. can you give an example? does your model have a default factory field that generates like a uuid each time?

Hey @Nate. Nothing generates a UUID. It does some date parsing into strings. I’m struggling to create a repro - only seems to happen in a live run. But knowing that Pydantic models should be fine will be a help - I’ll try to debug more in a day or so. Appreciate the reply.

👍

@Nate I’ve found a minimum reproducible example, which seems to have unexpected behaviour. Running it from the terminal a couple of times, it gives consistent hashes for the individual objects, but a different hash for the two together. It seems it might be the combination of two different types of object: i.e. a Pandas DataFrame with a Pydantic model. I’ve tried with two different DataFrames and it’s consistent, but breaks again when I re-add the Config object back in. I would love to be able to use the built-in caching policies rather than creating a custom

cache_key_fn

. Is this in scope for Prefect, or do you think it’s something else? Thanks for taking a look!

import pandas as pd
from prefect.utilities.hashing import hash_objects
from pydantic import BaseModel

class Config(BaseModel):
    active: bool = False

config = Config()
data = pd.DataFrame({"value": [True]})

print(f"data                 : {hash_objects(data)}")
print(f"config               : {hash_objects(config)}")
print(f"data, config         : {hash_objects(data, config)}")