yair friedman

    yair friedman

    3 months ago
    hi , i am executing my flow not from main flow. when i run it on the default executor everything is running file, when i use DaskExecutor it crashs with the following message 2022-06-22 14:06:40,649 prefect.TaskRunner INFO Task ‘train_model[0]’: Starting task run... 2022-06-22 14:06:40,650 prefect.TaskRunner INFO Task ‘train_model[1]’: Starting task run... The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. 2022-06-22 14:06:55.255 python[95140:4859256] * Terminating app due to uncaught exception ‘NSInternalInconsistencyException’, reason: ‘NSWindow drag regions should only be invalidated on the Main Thread!’ please advise.
    Kevin Kho

    Kevin Kho

    3 months ago
    Hey yair, so need to double post next time. We’ll see the message. This seems like a log from a task? This is not a Prefect log. What is the task doing?
    yair friedman

    yair friedman

    3 months ago
    i am running a ML task (internally it does all kind of scikit-learn transformes + run mljar-suprevised estimator) the task is mapped so multiple runs happens with the same unmapped data. the same flow runs without an issue with default executor , but fails with daskexecutor or LocalDaskExecutor.
    Kevin Kho

    Kevin Kho

    3 months ago
    Is this Prefect 1 or 2?
    yair friedman

    yair friedman

    3 months ago
    perfect 1
    Kevin Kho

    Kevin Kho

    3 months ago
    What does the mapped task return? A fitted model?
    Are you plotting with matplotlib?
    yair friedman

    yair friedman

    3 months ago
    return a dictionary of values
    return {"pipeline": pipeline_which_is_list_of_dict, "some_object":internal_object , "f1" :internal_object.metrics_dict['f1'] }
    i am not plotting directly. maybe matplotlib is used internally in the used packages
    Kevin Kho

    Kevin Kho

    3 months ago
    I really don’t know. There is a problem with parallelizing something inside your task. Might be some plotting like this. Maybe you can try processes for the LocalDaskExecutor to provide more isolation? But this will copy your DataFrame multiple times so it might bloat memory too
    yair friedman

    yair friedman

    3 months ago
    ok, will try to isolate the problem
    thanks
    ok, found the issue, it was with the mljar’s . when i switched to my own algorithm it all worked perfectly
    Kevin Kho

    Kevin Kho

    3 months ago
    Ah I see