Has anyone built a custom result storage class? I’...
# ask-community
m
Has anyone built a custom result storage class? I’m thinking about trying to make one for upstash Redis, where write will append the result to a list (sorted on time stamps) and it will always read from the end of the list. This way I can interact with results without going through prefect each time. Does that make sense or have I misunderstood Cache / Results?
👀 1
b
Hey Matt, just to clarify, what you're looking to do is persist the results of a prefect task or flow to a storage location, and continue to add results to this list going forward for each new flow/task run. Is that right? > This way I can interact with results without going through prefect each time. This part I'm a little confused on. Can you elaborate on the how you plan on interacting with the stored results, and why you don't necessarily want to use prefect?
custom result storage class
FWIW, i think blocks may help you out with this. you can create your own type of storage block too.
m
Thanks Bianca, I'll look into this and try to get a prototype up before you come online tomorrow. Essentially I'm looking to create a new block type with the following features • results are written to an append-only log for the given inputs, this creates an auditable trail that I can inspect without going through prefect. Typically for debugging purposes. I know you can persist results for every flow run, but I'm not sure if Prefect then allows you to partition the persisted results (eg. on a specific input parameter) or perform other database query-ish type work • using a distributed lock for a given input to ensure only a single task is responsible for computing a value for any given key. if concurrent task runs provide the same parameters, the other tasks will fail to acquire the lock & keep polling until result is available
ah actually, nevermind! I just discovered tags + concurrency limits, I think I can get everything done with this. the only awkward part will be that I need to dynamically generate the tag and ensure a concurrency limit of 1 before invoking the flow
🙌 1