Hi there - I have a function called `main` it runs...
# prefect-community
i
Hi there - I have a function called
main
it runs a
flow
. If I add a
@task
decorator to
main
is that an acceptable way to run a flow from a flow. or do you see any issues? ..
c
For purely local execution with Prefect Core I’m sure it will run just fine, but I’m not sure I see the benefit
i
Oh boy 🙂 I guess I dont understand some things. I understood from the documentation that to run a flow in production there needs to be a function wrapping
flow.run()
? And so if I have another flow that needs to run another flow then i should just call the wrapper function.
c
No worries! There’s nothing inherently wrong with it
I just don’t see the benefit of that pattern vs. including all of the tasks from the other flow into the main flow and just using a single big flow
j
@itay livni if you combined your flows you could use something like a trigger https://docs.prefect.io/core/concepts/execution.html#triggers on the first task of your “second” flow. e.g. make the first task of what would be your second flow only execute if the entire upstream is successful
Copy code
@task(trigger=all_successful)
def the_task():
    # do something interesting
    pass
i
@josh Hi - The two flows are separate programs if you will. They can be run independently or together. See non step function diagram. The
make nodes
function is a
mapped task
. Think of a knowledge graph. You build one node. It generates new nodes to make and so on. Does that make sense?
j
Ah I see, it’s a somewhat cyclical graph. Have you seen the looping mechanism in core? https://docs.prefect.io/core/examples/task_looping.html#task-looping This would be useful if you had a task that you want to keep iterating/checking over and over again until some condition is met
Throwing out some pieces which may help you write your Flows for your use case 🙂
i
Thank you. I looked at it. Not quite sure I understand it. What are the advantages to using loop over using a trigger?
Basically it's a recursive process that only breaks when max number of nodes in a knowledge graph are made. Making each node i.e.
make nodes
is (1) a complex process. (2) can be mapped\parallelized (?) once the first node is made.
j
Looping i would say is used in your use case of creating a flow from a task where the task could loop and run other flows until a specific condition is met. The trigger would be useful in the case you somehow combined these flows but that doesn't seem to be like the desired outcome.
This is an interesting dynamic case!
If the process breaks when the max nodes it made then it can use looping to run other flows until that happens
If you want to we can move to DMs or schedule a hangouts meet to discuss this use case!
i
@Chris White would love to do a hangout. Send over a time that is convenient for you.
c
@itay livni I think you meant to tag @Chris White?
😄 1
i
yes thanks!
👍 1