Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

:wave: peeking into the future a little bit for prefect (also apologies if this already exists and i missed it), do you think we might be able to specify different caching mediums (e.g. cache using a distributed cache like redis)?

e.g. if two flows running on different instances in my cluster happen to call the same task, it's not a big deal if they both run. if its 10,000 of the same call across a couple minutes, it would be awesome if i those results were stored in a distributed cache so that each task can just check said distributed cache

You can use blocks to store key-value pairs already, and if you want to use Redis, you can create (and even contribute) a Redis block, too

If this is a feature you’d like to see, feel free to open or follow an issue <https://github.com/PrefectHQ/prefect/issues|over on GitHub>.

Sorry for the delayed response but awesome! And yeah definitely will play around with blocks/open an issue if it feels like I can't figure it out :slightly_smiling_face:. Thanks so much!