hi, how can I start two tasks simultaneously? Now I see it’s running at a single thread, even two tasks have no dependencies, they cannot run at the same time. Is this even impossible?
k
Kevin Kho
10/25/2021, 3:19 AM
It’s possible if you use the LocalDaskExecutor or DaskExecutor, but we don;t support two simultaneous different mapped tasks. What executor are you on and what are you seeing?
t
Tony Yun
10/25/2021, 2:19 PM
I’m using Kube Runconfig. Yeah that’s what i think is right. It’s ok. I thought this is something native and easy to do but gave a lot of try still don’t get it working.
s
Sam Cook
10/25/2021, 3:36 PM
The Kube RunConfig is different from the executor. The default executor forces synchronous execution, but setting your flow to use the local dask executor will apply parallelism.
t
Tony Yun
10/25/2021, 3:43 PM
ok I haven’t used any other executor on Cloud other than Kube yet. Will try it sometime. Thanks!