@Marvin How do i timeout a workflow when the workflow has been idle rather than a hard time limit?

To handle an idle workflow timeout in Prefect, there isn't a built-in feature that directly supports an idle timeout. However, Prefect provides functionalities to pause or suspend a flow run, which might be utilized to build custom logic around idle timeouts: - Pausing a Flow Run: Prefect allows pausing an in-progress flow run with

pause_flow_run

and

resume_flow_run

functions. Paused flow runs time out after one hour by default, and you can specify a different timeout period in seconds using the

timeout

parameter. To implement an idle timeout, you could create a mechanism to periodically check if the workflow is idle and then call

pause_flow_run

if the conditions of being idle are met. After the specified idle timeout period, if the flow run hasn't been resumed (meaning it's still idle), it will fail. Here's a simple snippet on how to use the pausing functionality:

python
from prefect import flow, task, pause_flow_run

@task
def task1():
    # Task implementation

@task
def task2():
    # Task implementation

@flow
def my_flow():
    task1_res = task1()
    # Custom logic to check for idle condition
    if idle_condition:
        pause_flow_run(timeout=3600)  # Timeout after 1 hour if not resumed
    task2_res = task2()

my_flow()

The actual implementation of the

idle_condition

depends on how you define idleness in your workflow and may require you to track task execution times, flow state, and other metrics. Please refer to the parts of the Prefect documentation that cover pausing or suspending a flow run for more details.