Hello I am executing one flow in a remote distributed dask c

Question

Hello I am executing one flow in a remote distributed dask cluster Previously I executed my code locally and gathered the flow results in the local easy way e g `subflow res result deployed model result` When I wrote the code I read the docs knowing this way <https docs prefect io core idioms task results html|was not valid for remote executions> I would like to know the proper way to gather results when working in a remote distributed environment Can anyone help me

Kevin Kho · Answer

Hi < Antonio Manuel BR> what are you trying to do with the result Because the purpose of the remote execution is just to run the Flow

Antonio Manuel BR · Answer

I m iterating over a task containing that flow using LOOP signal in order to start the next iteration I need to save some results

Kevin Kho · Answer

Could you give me an example code You may have to persist the data inside the LOOP

Antonio Manuel BR · Answer

Here is the task containing the flow ` task` `def processMiniBatch ` `online data dir str ` `d model PrefectClassifier ` `c models List PrefectClassifier ` `n mini batches int ` `label encoder LabelEncoder ` `predictions dir str ` ` ` `loop payload = prefect context get task loop result ` `loop j = loop payload get loop j 1 ` `loop d model = loop payload get loop d model d model ` `loop c models = loop payload get loop c models c models ` `with Flow ` ` Process mini batch ` `executor = DaskExecutor f cluster ip 8786 ` ` as flow ` `deployed model = Parameter deployed model ` `contestant models = Parameter contestant models ` ` data filename = ingestData online data dir ` `is data tagged = checkDataTagged data ` `with case is data tagged False ` `predictions = predictData deployed model data ` `writePredictions predictions predictions dir filename ` `with case is data tagged True ` `data = dataPreparation data label encoder ` `contestant models = testModel map unmapped data contestant models ` `deployed model = testModel data deployed model ` `contestant models = trainModel map ` `unmapped data contestant models` ` ` `flow res = flow run ` `deployed model=loop d model contestant models=loop c models` ` ` `is data tagged = flow res result is data tagged result` `if is data tagged ` `loop d model = flow res result deployed model result` `loop c models = flow res result contestant models result` `if loop j == n mini batches ` `loop c models append loop d model ` `return loop c models` `loop j += 1` `raise signals LOOP ` `result=dict ` `loop j=loop j loop d model=loop d model loop c models=loop c models` ` ` ` `

Kevin Kho · Answer

You ll want to rip out the Flow into it s own Flow file register that and then the LOOP task can used the 3 tasks from Prefect `create flow run` `wait for flow run` `get task run result` as outlined in <https www prefect io blog prefect 0 15 0 a new flow run experience |this> blog

Antonio Manuel BR · Answer

I ll give it a try thank you very much Will write here for additional problems if needed