https://prefect.io logo
h

Henrik Väisänen

12/29/2020, 6:56 PM
Hey, could someone point, how to get the number of threads and cores Dask worker has within Prefect task? I'm not spawning them from prefect but using the existing workers
j

Jim Crist-Harif

12/29/2020, 7:12 PM
Sorry, are you asking how many threads/cores an individual dask worker has? Or how many threads/cores your whole cluster has?
Prefect tasks themselves take a single thread per active task in a worker.
h

Henrik Väisänen

12/29/2020, 7:15 PM
yes, the single dask worker that is currently executing the prefect task. There are multiple workers running on different hardware so I want to access this information.
j

Jim Crist-Harif

12/29/2020, 7:18 PM
For an accurate number (and assuming you're using a
DaskExecutor
) you could do:
Copy code
from distributed import get_worker
get_worker().nthreads
👍 1
That gives the number of threads the worker has running. To access the actual number of cores available on the machine (which might be different), you can use
os.cpu_count
h

Henrik Väisänen

12/29/2020, 7:20 PM
thanks.
get_worker
might be just what I need
4 Views