Chris Hart
04/06/2020, 8:15 PMJim Crist-Harif
04/06/2020, 8:33 PMdask.distributed.worker_client
. See the dask docs here: https://distributed.dask.org/en/latest/task-launch.html#connection-with-context-managerwith joblib.parallel_backend('dask'):
search.fit(digits.data, digits.target)
Chris Hart
04/06/2020, 8:35 PMJim Crist-Harif
04/06/2020, 8:35 PMMark Koob
05/09/2020, 12:26 PMdistributed.worker_client()
does give the correct behavior when launching a dask-ml
search from inside a task when running on a dask cluster!
@task
def search_model(searchcv, x, y):
with worker_client() as client:
return searchcv.fit(x, y)
Thanks Jim!