any one already made a redis Result class? I am us...
# ask-communityr
Rob Fowler
03/12/2021, 10:52 PMany one already made a redis Result class? I am using docker storage and I think maybe LocalResults to a nfs/gfs mount or make a redis one?
đź‘€ 1
Rob Fowler
03/14/2021, 8:45 AMWell that works really well. I am surprised that one is not in the product. Specially as an option for Docker storage. You want it @Chris White ?
d
Dylan
03/15/2021, 1:45 PM
@Rob Fowler We’d love it if you contributed a
RedisResultc
Chris White
03/15/2021, 2:51 PM
Yup, agreed — although I am very curious about the use case here; results are generally intended for “long form” storage (e.g., restart a flow run from failure after a day) or when infrastructure messes up, and a redis result won’t be able to offer that sort of functionality.
r
Rob Fowler
03/16/2021, 11:01 AMFinished it. The main use case for me is with Docker storage. With the redis helm chart you can have a very reliable/HA redis cluster that is still very fast. For me, I don't need the high availability, I want to be able to restart my workflows within a few minutes to hours. They are short but complex. I expose the set() ex expiry parameter so I set keys to expire after one day.
In truth, outside of a k8s cluster I have had redis instances running for literally years.
The code looks like a combo of the other Result handlers, trivial. I'll make a PR against prefect. I have been running it a few days just as an extension module.
Rob Fowler
03/16/2021, 11:02 AMThat said, it's not an unlimited storage option like S3, GCS and azure blobs. It is just a replacement if you are vibed out by LocalStorage.
c
Chris White
03/16/2021, 3:14 PM
Gotcha, makes sense!
r
Rob Fowler
03/17/2021, 12:33 AMI started a PR but please ignore it until I make a fix or two. The cloud serialisers are byte based but the redis interface is strings. I need to look at redis a bit more or choose another serialiser. I only found this because mypy complained. Which is good.
d
Dylan
03/17/2021, 1:03 PM
+1 to mypy
r
Rob Fowler
03/18/2021, 9:26 AMTrying to finish it off but I don't seem to be able to get mypy to know the configuration argument is filled in and impossible for it not to be.
Rob Fowler
03/18/2021, 9:28 AMhttps://github.com/PrefectHQ/prefect/blob/19f6adc8844a7d249ef49255ff6c89f6db038a5a/src/prefect/engine/results/redis_result.py#L36
src/prefect/engine/results/redis_result.py36 error: Argument 1 to "from_url" of "Redis" has incompatible type "Optional[str]"; expected "str"
Rob Fowler
03/18/2021, 9:28 AMOpen to suggestions. For some reason the azure blob code that does the same thing passes.
Rob Fowler
03/19/2021, 6:32 AMfix that bit, next is to look at the redis.get returned types and I'm done. I know redis get and set are binary strings but I think they are stringed somehow.