hi , i am executing my flow not from main flow. when i run it on the default executor everything is running file, when i use DaskExecutor it crashs with the following message 2022-06-22 140640,649 prefect.TaskRunner INFO Task ‘train_model[0]’: Starting task run... 2022-06-22 140640,650 prefect.TaskRunner INFO Task ‘train_model[1]’: Starting task run... The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. 2022-06-22 140655.255 python[95140:4859256] * Terminating app due to uncaught exception ‘NSInternalInconsistencyException’, reason: ‘NSWindow drag regions should only be invalidated on the Main Thread!’ please advise.

Hey yair, so need to double post next time. We’ll see the message. This seems like a log from a task? This is not a Prefect log. What is the task doing?

i am running a ML task (internally it does all kind of scikit-learn transformes + run mljar-suprevised estimator) the task is mapped so multiple runs happens with the same unmapped data. the same flow runs without an issue with default executor , but fails with daskexecutor or LocalDaskExecutor.

Is this Prefect 1 or 2?

perfect 1

What does the mapped task return? A fitted model?

return a dictionary of values

return {"pipeline": pipeline_which_is_list_of_dict, "some_object":internal_object , "f1" :internal_object.metrics_dict['f1'] }

I really don’t know. There is a problem with parallelizing something inside your task. Might be some plotting like this. Maybe you can try processes for the LocalDaskExecutor to provide more isolation? But this will copy your DataFrame multiple times so it might bloat memory too

ok, will try to isolate the problem

Ah I see