Hello - I am mostly deployed and things are working well - running prefect server with DaskExecutor. I have created a custom wrapper around the Flow class so that I can easily set the run environment for local development vs production flows.I have a function that is creating the dask scheduler/worker through dask gateway - all works, EXCEPT that the dask cluster is also spun up on registration. Is there an easy way to prevent this? I am wondering if I have to make a custom dask class and use that as part of the executor, rather than a custom function that creates it as part of initiating the flow.
yeah I have one example showing this with Fargate:
from dask_cloudprovider.aws import FargateCluster # or ECSClusterpredeffargate_cluster(n_workers=4):
"""Start a fargate cluster using the same image as the flow run"""return FargateCluster(n_workers=n_workers, image=prefect.context.image)
flow.executor = DaskExecutor(