Alvis Tang
02/04/2021, 9:54 PMrun_agent
but I can't make it work
I tried to use
flow.executor = LocalDaskExecutor()
flow.run()
& it can run in parallel, but not run_agent
. any clue?Zanie
run_agent
it's just going to be confusing. I'd recommend starting the agent in a separate process. Your flow script should have flow.register()
and then you'd run the flow from the UI and run an agent in a separate process to kick off the flow run. Either way, using the dask executor should be giving you the parallelism you want.Alvis Tang
02/04/2021, 10:05 PMprefect agent local start
a tryprefect agent local start
I can run 4 tasks in parallel. Is there anyway I can adjust this number? I have a HPC so it'd be great if I can run more at the same timeZanie
Alvis Tang
02/04/2021, 11:02 PMZanie
Alvis Tang
02/04/2021, 11:15 PMLocalDaskExecutor(_num_workers_=n_workers)