https://prefect.io logo
a

Adrien Boutreau

02/01/2021, 3:17 PM
Hello guys! I'm using this example : https://docs.prefect.io/core/examples/functional_docker.html and I would like to know if it's possible to add a retry if python job fail - I saw how to do it with a task but not CreateContainer function
j

Josh Greenhalgh

02/01/2021, 3:22 PM
CreateContainer
is a subclass of
Task
which takes arguments related to retries - so you should just be able to pass those down?
k

Kyle Moon-Wright

02/01/2021, 5:19 PM
Yup you can pass those kwargs when instantiating the task - using that doc as an example:
Copy code
from datetime import timedelta

container = CreateContainer(
    image_name="prefecthq/prefect",
    command='''python -c "from prefect import Flow; f = Flow('empty'); f.run()"''',
    max_retries=3,
    retry_delay=timedelta(seconds=2)
)
a

Adrien Boutreau

02/01/2021, 8:02 PM
thanks for your feedbacks @josh @Kyle Moon-Wright
So with your commands, Prefect will try X time to create container, but if command line failed, it didn't retry
any idea on how I can solve it ?
k

Kyle Moon-Wright

02/01/2021, 8:19 PM
Hmm, well the retry will kick in if the State of the task has ended in failure. In this case, the command may have succeeded in executing even though the container creation failed..? I certainly expected the CreateContainer failure to bubble to the task - does that seem to be the case here? edit: Hmm I’m unable to reproduce, but we may need to separate the command logic part specifically.
Also, I’d recommend using
image_name="prefecthq/prefect:latest"
if you’re pulling from our registry
a

Adrien Boutreau

02/01/2021, 8:37 PM
did you
Copy code
command='''python -c "from prefect import Flow; f = Flow('empty');
by making
Copy code
command='''python -c "from prefect2222 import Flow; f = Flow('empty');
making an import of something which is creating an error ? On my side, it will try only once and exit
failed but no retry 😢
Ok found it
status_code = WaitOnContainer(max_retries=5, retry_delay=timedelta(seconds=1))
k

Kyle Moon-Wright

02/01/2021, 8:42 PM
Gotcha, I was just looking at the WaitOnContainer task. It’s working as expected with the retry logic there instead?
a

Adrien Boutreau

02/01/2021, 8:42 PM
yes!
success kid 1
🎉 1
thanks a lot for your help
k

Kyle Moon-Wright

02/01/2021, 8:44 PM
I gladly accept the credit, and none of the blame…
Nice work! Thanks for sharing.