This runs with line 8 commented out, but raises th...
# ask-communityn
Nate Atkins
02/07/2020, 7:59 PMThis runs with line 8 commented out, but raises the KilledWorker when it is run.
j
Joe Schmid
02/07/2020, 8:03 PM@Nate Atkins Maybe your Dask workers don't have the
requestsrequestsn
Nate Atkins
02/07/2020, 8:07 PMI thought about that. I checked to make sure I was starting the workers in the venv and directory that the working example runs in when I let the executor start the cluster.
It also seems like commenting out the requests.get() would still blow up as the model is still loaded. Maybe this isn't the case as I don't fully understand what is getting passed from the scheduler to the worker. I can check this by removing the package and see what happens.
Nate Atkins
02/07/2020, 8:15 PMThis is what I see in the dask-worker output.
[2020-02-07 201343,099] INFO - prefect.TaskRunner | Task 'retrieve_url': Starting task run...
objc[68582]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[68582]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
distributed.nanny - INFO - Worker process 68582 was killed by signal 6
distributed.nanny - WARNING - Restarting worker
Nate Atkins
02/07/2020, 8:19 PMGot it! https://github.com/darkskyapp/forecast-ruby/issues/13
On MacOS Mojave
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
j
Joe Schmid
02/07/2020, 8:21 PM@Nate Atkins Nice, great find!
c
Chris White
02/07/2020, 9:23 PM
@Marvin archive “Unexpected error occured in FlowRunner: KilledWorker”
m
Marvin
02/07/2020, 9:23 PM2 Views