https://prefect.io
Join Slack
How do Retries work with Dependent Flows? I added retries to the task from the parent flow, but the...
# ask-community
m
How do Retries work with Dependent Flows? I added retries to the task from the parent flow, but they don't seem to be firing?
k
Does the subflow not fire? Are you trying to fire another subflow?
m
I'm trying to get the subflow to retry if it fails (specifically, if the EC2 container doesn't start - occasionally get the error 
'reason': 'RESOURCE:MEMORY')
k
Ah ok so this can’t be done easily. See this for an explanation. Basically the idempotency will not fire off another run
Sub-flow retries are being worked on though for 2.0
m
aha
Looks like it could work if I used an Idempotency Key and used script-based storage?
How do you use Script-based Storage with S3 storage?
k
Just use 
stored_as_script=True
and point to a 
local_script_path
to upload. Use them in tandem.
m
Hrm, I'll give that a try, thanks!
@Kevin Kho How does the path work? So let's say my CI/CD pipeline runs 
prefect register <path>
- do I put 
<path>/<script>.py
? Or just 
<script>.py
?
k
I think relative will work
👍 1
You should see it upload for you
m
Huzzah! btw, needed to use the absolute path
Sweet, now let's see if the Idempotency key stuff works
k
Ah ok that also makes sense
11 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by