Hi, I was wondering if there were any plans to develop Prefect further into the space of durable execution? Meaning adding some of the capabilities that Temporal holds in which a workflow can be ran forever, even if the worker itself crashes, it can recover.

hi @oeharel - the prefect worker today when running on kubernetes via the helm chart or as a process managed by systemd (for example) will recover, provided there's not a fundamental issue with the infrastructure do you mean something else beyond this?

Actually that's interesting. Does this mean that if the worker it self crashes during a flow. It will recover and won't run any previous flows? Beyond that, Temporal has the "reset" feature which allows to start a flow from a certain step - this could be an interesting feature for Prefect.

> Does this mean that if the worker it self crashes during a flow. It will recover and won't run any previous flows? no it doesn't, if you have N flows that move from

SCHEDULED

->

PENDING

(because say its now their scheduled run time), and then the worker notices and picks up the first one but then crashes you'd have N - 1 flow runs in

PENDING

(which would get picked up as soon as there's a healthy worker to do so) and 1

CRASHED

flow run

Say I have the following:

@flow
async def main_run():
  await subflow_1()
  await subflow_2()
  await subflow_3()

If

subflow_2

crashes (not only due to a system crash), what reset it Temporal basically allows me to do is start

main_run

from

subflow_2

meaning it'll still continue to

subflow_3

. To be more technical, they create events for every "landmark" in a run (which from my familiarity Prefect sort of has): 1. Flow scheduled 2. Flow started 3. Flow completed 4. ..... Then when you reset you can tell it to start from any of these events. Very useful feature :)

gotcha! yeah you do this with

Results

and retries in prefect if subflow 1 and 2 finish in completed and persist a result, then at any point in the future you could retry that

main_run

and start from the first non-completed state

Ahaaaaa, thank you very much! This shpuld be advertised better, I am not sure I saw this in the docs