Ying Ting Loo
07/18/2024, 6:25 AM@task(retries=1, retry_delay_seconds=900, task_run_name="{dbt_command}")
def wrapper_dbt_run(dbt_command, dbt_cli_profile, workdir):
return trigger_dbt_cli_command(
dbt_command,
dbt_cli_profile=dbt_cli_profile,
overwrite_profiles=True,
helper_command=f"cd {workdir}",
return_all=True,
)
runflow = wrapper_dbt_run.submit(
"dbt run",
dbt_cli_profile=dbt_cli_profile,
workdir=workdir,
wait_for=[runb],
)
will the task retry when it fails?Marvin
07/18/2024, 6:25 AMretries
and retry_delay_seconds
parameters you set in the @task
decorator, even when you use task.submit
to run the task. This means that if the task fails, it will be retried once after a delay of 900 seconds (15 minutes) as specified in your code.
For further information, you can refer to the Prefect documentation on tasks:
https://docs.prefect.io/latest/concepts/tasks/Marvin
07/18/2024, 6:26 AMtask.submit
. The retry behavior specified in the @task
decorator will be respected even when the task is submitted using submit
.
Here is an example from the documentation that demonstrates setting retry parameters:
python
from prefect import task
from prefect.tasks import exponential_backoff
@task(
retries=3,
retry_delay_seconds=exponential_backoff(backoff_factor=10),
retry_jitter_factor=1,
)
def some_task_with_exponential_backoff_retries():
pass
You can read more about this in the Prefect documentation on tasks:
https://docs.prefect.io/latest/concepts/tasks/
Additionally, here is a basic example of setting task retries:
https://raw.githubusercontent.com/zzstoatzz/prefect-code-examples/main/examples/tasks/basic.py