Hello Everyone, I have 2 questions. 1. Can we have more than 1 DAGs running concurrently in the same environment ? 2. Can we have executing different instances of same DAG running concurrently, with each instance having different parameters ?

Hi @Aakarsh Nadella: 1. depends on what you mean by “environment”; you can of course call

flow.run()

in as many different places / processes as you’d like 2. similar to the above, you can call

flow.run(parameters={})

in many parallel python processes without any issues I suspect you’re asking a deeper question about orchestration of workflows / kicking off flow runs asynchronously via some sort of API, etc. in which case I’d recommend you take a look at Prefect Cloud (https://docs.prefect.io/cloud/faq.html), but let me know if I am misunderstanding your questions!

Hi @Chris White: So you mean we can run "n" number of flows of same DAG and each flow with different set of parameters ?

yea sure! Unless I’m missing some nuance of your question, this would work:

python my_flow_file.py &
python my_flow_file.py &
python my_flow_file.py &
...

and you could parse command line arguments as parameter values if you’d like

Gotcha ! But can we run all the command line arguments mentioned above at once ? Looks like when we enter the the first argument, it starts execution and we can execute the second argument (with different parameters) only after the first one completes

Yea the

&

should run each process in the background, but that can be platform specific (you might need to use

&&

,

nohup

or

screen

to background the processes, depending on your OS)

Nice. Thanks for the information. Could you please share me the documentation of using

&

for executing concurrent processes

Thanks Chris.