Philipp Eisen
01/19/2022, 12:02 AMConsider scattering large objects ahead of time
with client.scatter to reduce scheduler burden and
keep data on workers
future = client.submit(func, big_data) # bad
big_future = client.scatter(big_data) # good
future = client.submit(func, big_future) # good
I was wondering, what would be the prefect pattern here to scatter the object ahead of time?Kevin Kho
Philipp Eisen
01/19/2022, 9:36 AMKevin Kho