https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> flow foo is running task bar in parra...
# marvin-ai
m
@Marvin flow foo is running task bar in parralel using a DaskTaskRunner, can I use retries lwithin the dasktaskrunner like any other task? if yes, what are the relevant parameters and where to set them?
m
✅ thought for 121.6 seconds
Yes — Prefect task retries still work when the task is executed via 
DaskTaskRunner
. The important bit is: retries are configured on the task, not on 
DaskTaskRunner
. Example:
Copy code
python
from prefect import flow, task
from prefect_dask import DaskTaskRunner

@task(
    retries=3,
    retry_delay_seconds=10,
)
def bar():
    ...
    
@flow(task_runner=DaskTaskRunner())
def foo():
    bar.submit()
Relevant retry parameters on 
@task
 - `retries`: how many times to retry after failure - `retry_delay_seconds`: delay before retrying - can be a single number - can also be a list of delays - can also be generated dynamically in some cases - `retry_jitter_factor`: adds jitter to retry delays - `retry_condition_fn`: custom function to decide whether a failure should actually be retried Example with more options:
Copy code
python
from prefect import flow, task
from prefect_dask import DaskTaskRunner

def should_retry(task, task_run, state):
    return True

@task(
    retries=5,
    retry_delay_seconds=[1, 5, 10, 30, 60],
    retry_jitter_factor=0.5,
    retry_condition_fn=should_retry,
)
def bar():
    ...
    
@flow(task_runner=DaskTaskRunner())
def foo():
    futures = [bar.submit() for _ in range(10)]
Where to set them - Set them on the task definition with 
@task(...)
- Optionally, Prefect also has defaults for task retries via settings, but the main place is still the task itself What not to configure - 
DaskTaskRunner
itself does not have special retry parameters for Prefect task retries - Dask is handling parallel execution; Prefect is handling task state/retry behavior So in your case: - 
foo
uses 
DaskTaskRunner
for parallelism - 
bar
can use retries exactly like any other Prefect task - configure retries directly on 
bar
Docs / package: - Prefect docs - prefect-dask docs If you want, I can also show: 1. a full example with 
submit()
+ retries + waiting on futures, or 2. how flow retries differ from task retries in this setup.
m
@Marvin what does retry_jitter_factor do
m
✅ thought for 7.1 seconds
retry_jitter_factor
adds randomness to your retry delay so that retries don’t all happen at the exact same moment. Why that helps: - if many parallel tasks fail together - and all have 
retry_delay_seconds=10
- then without jitter they all retry again at 
t+10s
- that can create another spike against your API / DB / service With jitter, Prefect spreads those retries out a bit. Example:
Copy code
python
@task(
    retries=3,
    retry_delay_seconds=10,
    retry_jitter_factor=0.5,
)
def bar():
    ...
Here, the base retry delay is 
10
seconds, and Prefect will apply some randomness around that delay so retries are staggered instead of synchronized. This is especially useful with: - 
DaskTaskRunner
- many concurrent tasks - rate-limited APIs - flaky downstream services So conceptually: - 
retry_delay_seconds
= base delay - 
retry_jitter_factor
= how much randomness to add to that delay If you want, I can also show a concrete example of the actual delay range Prefect uses for a given jitter factor.
m
@Marvin yes
m
✅ thought for 84.2 seconds
Yep — in Prefect 3.x, 
retry_jitter_factor
means the retry delay is randomized around the base delay, not just increased. So for:
Copy code
python
@task(
    retries=3,
    retry_delay_seconds=10,
    retry_jitter_factor=0.5,
)
def bar():
    ...
the retry delay will be a random value in roughly this range: - minimum: about 
6.29s
- maximum: 
15s
So it can be: - shorter than 10 seconds - longer than 10 seconds It does not just add up to 50% extra delay. It jitterizes the delay within a bounded range. Concrete example With 
retry_delay_seconds=10
and 
retry_jitter_factor=0.5
, one retry might happen after: - 
6.4s
- 
8.6s
- 
9.9s
- 
11.2s
- 
13.0s
- 
14.8s
all valid possibilities. Why this exists If 100 Dask-backed tasks fail at once and all retry after exactly 
10s
, they stampede the downstream service again. Jitter spreads them out. Rule of thumb - 
retry_delay_seconds
= target delay - 
retry_jitter_factor
= how widely Prefect can randomize around that target If 
retry_jitter_factor
is: - 
0
or unset: retry happens at exactly the base delay - `0.5`: retry is randomized in a bounded window around the base delay - larger values: wider spread One subtle detail: Prefect uses a bounded distribution, so the spread is not perfectly symmetric, but the average delay stays close to the base value. If you want, I can also show a tiny table for: - 
jitter_factor=0.1
- 
0.5
- 
1.0
so you can see the approximate retry ranges side by side.
Open in Slack
PreviousNext
Powered by