anyone have luck setting

PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS

with dask-gateway? We use a dynamic cluster per flow, rather than static, so we can’t fill in an exact address, as it depends on the cluster that has spun up as part of the flow running. My goal is to have a helper file that configures the default executor and runconfig for a user so they don’t have to worry about their flow’s configuration.

Hey @Sam Werbalowsky, I haven’t seen anyone try to set this. I have seen some people subclass

Flow

and then add their executor and runconfig so they don’t have to repeatedly do it. Maybe you can do the same but have it take inputs?. If you have a function, you can also set it that way? What issue are you running into? If you feel that the executor is not being respected, it may very well be because the executor is not serialized when a flow is registered. The executor is looked for in the flow file when the flow is loaded from storage. Because of this, it can not be set properly if the executor is not in the file.

Hey hey, yeah so basically I want to have a library (let’s call it helpers that has something like this…):

class MyFlow(Flow):
    self.config = config()

def config():
   if LOCAL_DEVELOPER_MACHINE:
      return LocalExecutor()
   else:
      gateway = Gateway()
      cluster = gateway.new_cluster()
      return DaskExecutor(cluster.stuff)

I figured I could avoid this by just using the defaults.

This will work, just that you need helpers available during execution time. I personally haven’t seen the default class set

Gotcha. Yep, that shouldn't be a problem in our end. Thanks!