Hey folks, is it possible when running a script wi...
# ask-communityf
Federico Zambelli
04/21/2023, 10:00 AMHey folks, is it possible when running a script with prefect, to specify which flow to run?
E.g. imagine this:
Copy code
@flow
def first_flow():
...
@flow
def second_flow():
...
def main():
first_flow()
second_flow()d
Deceivious
04/21/2023, 10:05 AM Copy code
if <cond1>:
first_flow()
if <cond2>:
second_flow()f
Federico Zambelli
04/21/2023, 10:06 AMyeah this I imagined was the poor man's solution. I was wondering if prefect had something built-in for this
d
Deceivious
04/21/2023, 10:08 AMI would maybe rename main and make it a flow and trigger the other 2 flows as sub flows if you only want a single deployment.
But cant say more without more context on why that would be
a poor man's solutionf
Federico Zambelli
04/21/2023, 10:09 AMyeh sorry, poor man's solution probably was not the proper term.
Lemme illustrate better
Federico Zambelli
04/21/2023, 10:12 AMImagine you have a task (or a bunch of) that return unpickleable objects, and therefore can't be cached. This is my current case, as i have a couple of tasks that return async generators.
Eventually in the first flow I persist the result of these generators to disk (first flow), and later do something else with this results, reading from disk (2nd flow)
I would like to, say, skip the execution of first flow if input parameters are the same as the last ones used, for instance.
d
Deceivious
04/21/2023, 11:16 AMYour first task could be
Copy code
def generate_or_load_items(): #just an example
if file.exists():
return load_items()
return generate_items()f
Federico Zambelli
04/21/2023, 11:17 AMalright fair enough, thanks for the help!
🙌 1
2 Views