And another complete different question 🙋 I have set up the LocalDaskExecutor (via the env variable) and my flow is run by the cloud. However, tasks are run one by one. How can I achieve parallelism and run more than one in parallel?

The executor that your Cloud Flow run uses is determined by the execution environment it has: https://docs.prefect.io/orchestration/execution/overview.html#environments In your case, you will probably want to use a

RemoteEnvironment

with the

LocalDaskExecutor

configured as in the examples above

I ran into this resent with the KubernetesJobExecutor because I needed to specify the job_spec yaml, short story is the default LocalDaskExecutor wants to use synchronous execution, and there's no way for you to specify it to use threads or process.

Ah I see — yea there isnt a good way to configure it via environment variable, but the

RemoteEnvironment

allows you to pass

executor_kwargs

(in this case

scheduler="threads"

)

I wrote a basic extension to the KubernetesJobExecutor to allow you to specify the executor and kwargs like the RemoteEnviroment allows, and it allowed you to get the paternalism

Thanks all for your answers!