Jason Raede
12/28/2021, 1:58 AMLocalAgent
can run using the LocalDaskExecutor
? It’s parallelizing stuff fine locally when I just run flow.run
but on a server running a local agent it looks like it’s going one task at a time (I’m just watching the logs in prefect cloud). I’m setting flow.executor
before registering it, if that matters. I don’t see any indication in Prefect Cloud of what the flow’s executor is.Kevin Kho
n_workers
explicitly to make sure it uses the available cores.Kevin Kho
num_workers
.
flow.executor = LocalDaskExecutor(scheduler="threads", num_workers=8)
taken from hereJason Raede
12/28/2021, 4:22 AMKevin Kho
Jason Raede
12/28/2021, 4:04 PMJason Raede
12/28/2021, 4:04 PMKevin Kho
Kevin Kho
Jason Raede
12/29/2021, 2:31 AM