Is it possible to set max retries on

Question

Is it possible to set max retries on <https docs prefect io api latest core task html task 2|Tasks> from a Parameter i e runtime

Anna Geller · Answer

there is not I had an example using KV Store for that in the past but this was an extremely hacky solution to an extremely loosely defined problem smile Can you explain why would you need that

Anna Geller · Answer

you could e g retry only on specific exception types using <https github com PrefectHQ prefect pull 5634|retry on>

Martin T · Answer

This task checks for data to be ready in an external source using `max retries` and `retry delay` in combination with `raise signals FAIL SUCCESS` The flow is spawn from a main wrapper flow that can pass `Parameter ` s on The optimal values for `max retries` `retry delay` are not known up front and will likely be adjusted several times I don t like hardcoding these and having to release a new flow version to test other values

Anna Geller · Answer

thanks for explaining your use case this helps to check whether data arrives you would need to raise a RETRY signal rather than using retry from a parameter task check the first section of this <http ype 20 help 20 copyright 20 credits 20or 20 license 20for 20more 20information 20> gt gt 20class 20A 20 20 20 20 20 20def 20my method self 20x 20 20 20 20 20 20 20 20 20 20 20 20 20 20self setattr foo 20x 20 20 20 gt gt gt 20class 20B 20 20 20 20 20 20pass 20 20 20 gt gt gt 20my b 20= 20B 20 gt gt gt 20A my method my b 20 bar 20 gt gt gt 20my b foo 20 bar 20I 20guess 20it 20makes 20sense 20since 20self 20is 20just 20an 20arg 20being 20passed|Discourse topic gt with a full example

Martin T · Answer

I tried to `raise signals RETRY` but that doe not respect `max retries` so I use `FAIL` at the moment

Martin T · Answer

< Anna Geller> discourse link

Anna Geller · Answer

oh something went wrong with the link sorry <https discourse prefect io t is there an equivalent to sensors in prefect how do i trigger event driven workflows 76>

Anna Geller · Answer

```import pendulum from prefect engine signals import RETRY import awswrangler as wr def check if file arrived in s3 return wr s3 does object exist <s3 bucket example file csv> task def s3 sensor kwargs bool s3 object arrived = check if file arrived in s3 if bool s3 object arrived is False raise RETRY File not available yet retrying in 20 seconds start time=pendulum now add seconds=20 ```

Martin T · Answer

Okay this is similar to my case I did not know of `start time` option This is a very unfortune thing about ` args kwargs` documentation style neutral face <https docs prefect io api latest engine signals html retry> Even now that you tell me I cant see what other options i can pass to `RETRY ` I can only find the state constructor by reading source code

Martin T · Answer

My first guess is is this state constructor but that does not take any `start time` or `args` <https docs prefect io api latest engine state html state 2>

Anna Geller · Answer

thanks for the feedback on docs Are you good with your original issue

Martin T · Answer

Yep I can solve this via `start time` and `run count` slightly smiling face <https docs prefect io api latest engine state html retrying>