Hello, I am using Prefect-dask: DaskTaskrunner for concurrent task processing.
But somehow it always runs 2 task concurrently. Other 5 task are in pending stage. Is it because of the resource of the server or do I need to change any configuration?
Thanks in advance
01/12/2023, 3:25 PM
Hi @Mrityunjoy Das - I think this needs a little bit more detail to properly say.
How many resources are available?
Are you using a local dask cluster, or connecting to an existing dask cluster?
what does your dask configuration look like for # workers / threads?
How are you submitting the work to dask (e.g. the actual task code doing the submission)?