https://prefect.io logo
Title
m

Matt Alhonte

06/16/2022, 11:02 PM
How do Retries work with Dependent Flows? I added retries to the task from the parent flow, but they don't seem to be firing?
k

Kevin Kho

06/16/2022, 11:11 PM
Does the subflow not fire? Are you trying to fire another subflow?
m

Matt Alhonte

06/16/2022, 11:14 PM
I'm trying to get the subflow to retry if it fails (specifically, if the EC2 container doesn't start - occasionally get the error
'reason': 'RESOURCE:MEMORY')
k

Kevin Kho

06/16/2022, 11:15 PM
Ah ok so this can’t be done easily. See this for an explanation. Basically the idempotency will not fire off another run
Sub-flow retries are being worked on though for 2.0
m

Matt Alhonte

06/16/2022, 11:17 PM
aha
Looks like it could work if I used an Idempotency Key and used script-based storage?
How do you use Script-based Storage with S3 storage?
k

Kevin Kho

06/16/2022, 11:58 PM
Just use
stored_as_script=True
and point to a
local_script_path
to upload. Use them in tandem.
m

Matt Alhonte

06/16/2022, 11:59 PM
Hrm, I'll give that a try, thanks!
@Kevin Kho How does the path work? So let's say my CI/CD pipeline runs
prefect register <path>
- do I put
<path>/<script>.py
? Or just
<script>.py
?
k

Kevin Kho

06/17/2022, 12:05 AM
I think relative will work
👍 1
You should see it upload for you
m

Matt Alhonte

06/17/2022, 12:24 AM
Huzzah! btw, needed to use the absolute path
Sweet, now let's see if the Idempotency key stuff works
k

Kevin Kho

06/17/2022, 12:25 AM
Ah ok that also makes sense