hi <@U02H031QSN5>, came across this message from last month and am curious if there's an updated tim...

Brett Naul

09/23/2022, 2:49 PM

hi @Jean Luciano, came across this message from last month and am curious if there's an updated timeline for this feature (caching task results to somewhere other than local storage); am I interpreting correctly that there's currently no support for caching between flow runs? or really even for the same run but in any distributed context where there's no shared local filesystem?

👀 1

✅ 1

Brett Naul

09/23/2022, 2:51 PM

cc also @Christopher Boyd I see your message here https://prefect-community.slack.com/archives/CL09KU1K7/p1663681075409169?thread_ts=1663658867.329879&cid=CL09KU1K7

If you’d like to persist results for more than that (e.g. across flows), it would need to be persisted to remote storage.

do you mean this is already supported, or just that in theory you could add this logic yourself to all your tasks?

Brett Naul

09/23/2022, 2:55 PM

just noticed https://github.com/PrefectHQ/prefect/pull/6908 I guess all of this is pretty WIP still?

Zanie

09/23/2022, 3:04 PM

I'm hoping to move pretty quickly on the remaining work. It's our highest priority right now.

👍 2

🙏 1

Brett Naul

09/23/2022, 3:06 PM

sweet, looking forward to it 🙂

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.