I have 3 flows for a data workflow (flow1=extracts the data, flow2= makes data ready, flow 3= creates training set)
For the next step of work, we want to run this workflow (3 flows) in parallel for different parameters +1000 times .
Our concerns are:
1- memory
2- how to set up the pipeline to run +1000 flows in parallel + run the subflows sequentially
Any thoughts?