<@ULVA73B9P> can i increase the seconds to retry b...
# ask-marvin
o
@Marvin can i increase the seconds to retry based on a http response code? currently my task is decorated by @task(retries=5, retry_delay_seconds=exponential_backoff(backoff_factor=2)) one idea would in case of 5xx responses to not consume the 5 retries or reset them, another idea would be to increase retry dely seconds by several hours. what is the community way to handle two different scales of outages with task retires?
m
thought for 359.3 seconds
Short answer: you can decide whether to retry based on the HTTP response/exception, but you can’t change the delay schedule on-the-fly per error. In Prefect 3.x the retry delays are computed from
retry_delay_seconds
up front; there’s no built-in way to “reset” or “not consume” a retry once a run has failed. The community pattern for “two different outage scales” is: - Use
retry_condition_fn
to choose which failures are worth retrying (e.g., retry on 5xx/429/timeouts; stop immediately on most 4xx). - Provide a delay schedule that mixes short early retries with longer later retries (precomputed as a list or via a callable). - Add jitter to avoid thundering herd. Example
Copy code
from prefect import task
from prefect.tasks import exponential_backoff
import httpx

def retry_on_http(task, task_run, state) -> bool:
    """
    Retry strategy:
    - Retry 5xx and 429 (server/rate-limit)
    - Don't retry most 4xx (client errors)
    - Retry common network/timeouts
    """
    try:
        state.result()
    except httpx.HTTPStatusError as exc:
        code = exc.response.status_code
        if 500 <= code < 600 or code == 429:
            return True
        # stop on other 4xx
        return False
    except (httpx.ConnectError, httpx.ReadTimeout, httpx.WriteTimeout):
        return True
    except Exception:
        # default: retry
        return True
    return False

def two_scale_delays(n: int) -> list[float]:
    """
    First few retries are quick, then switch to multi-hour delays.
    Adjust to taste.
    """
    fast = [30, 120, 300]              # 30s, 2m, 5m
    slow = [3*3600] * max(n - len(fast), 0)  # then 3h each
    return (fast + slow)[:n]

@task(
    retries=6,
    retry_delay_seconds=two_scale_delays,      # or use a fixed list
    retry_jitter_factor=0.3,                   # optional: add jitter
    retry_condition_fn=retry_on_http,
)
def call_api(url: str):
    r = httpx.get(url, timeout=10)
    r.raise_for_status()
    return r.json()
Notes - You can’t reset retry count or “not consume” a failed attempt, but
retry_condition_fn
can stop further retries for permanent errors so you don’t waste attempts. - If you have a specific Retry-After from 429, Prefect can’t dynamically set the next delay to match it; approximate by including longer delays in your schedule and using the condition function to continue retrying only when appropriate. Docs - Task retries how-to - Tasks concepts - Tasks API reference If you share the exact services/status codes you care about (and preferred short/long delays), I can tailor the condition and delay schedule precisely.