Hey everyone, I'd like to start a `FlowRunTask` ba...
# prefect-community
l
Hey everyone, I'd like to start a
FlowRunTask
based on the result of a previous task. To be precise: Let's say I have a task and that task needs to fetch data from an API but hits the API request limit, now I want the task to return the data that it was able to fetch but also some sort of flag that shows that the task couldn't fetch all data because the api limit was hit, e.g. in a dict
{"data": data, "all_data_fetched": False}
. Now if
all_data_fetched
is false I'd like to start a new flow that continues fetching the data one hour later since that's the time it takes until the api limit is reset. I want to create a new flow since we're running on fargate and I don't want to let the task wait for one hour and pay all the time. And at the same time I want the rest of my flow to continue processing that data that the task was able to fetch from the API. I hope a) it's somewhat understandable what I want to achieve, b) it's possible and c) there is someone out there who can give me a hint on how to implement it. My biggest issue is that I don't know how to access the result from a task within a flow to check if I need to trigger the
FlowRunTask
or not. AFAIK I cannot run
FlowRunTask
from within a task, right? TIA 🙂
d
Hi @Lukas, What you’re proposing is definitely possible with Prefect! Check out: https://docs.prefect.io/api/latest/tasks/control_flow.html#control-flow-tasks
One of these should allow you to say “if the result of task x is foo, do task y (flow run task) with parameters bar and scheduled start time baz”
l
Hi @Dylan, thanks for your reply! I'm somewhat familiar with the control-flow-tasks but what if my flow returns a
dict
with multiple values and not just one boolean, how can I tell the control-flow-task to use a
dict
value as comparison value?
d
I would insert an additional task that did the “thinking”
i.e. accepts the
dict
as a parameter and returns a boolean
then your control flow task is straightforward
l
Makes sense, I'll try it this way 👍 Thanks a lot!
d
Anytime!
Happy Engineering 😄
marvin 1
🚀 1