I’m trying to start a cluster with a given worker image on dask-gateway, via prefect. How should I specify the image pull secret for the image?

executor = DaskExecutor(
    address="...",
    cluster_class="dask_gateway.GatewayCluster",
    cluster_kwargs={
        image="my/image/...",
        image_pull_secret="..." # ???
    },
    adapt_kwargs={minimum=2, maximum=10}
)

Hi @Shaun Cutts -

cluster_kwargs

are passed directly to the Dask cluster object so aren't within the Prefect purview. If you're looking for Prefect-specific image pull secrets, you'll want to specify those on the environment instead. For more info on that, take a look at https://docs.prefect.io/orchestration/execution/dask_k8s_environment.html#overview

Ah — ok. I assume that Dask Gateway needs the pull secret, so specifying in the environment won’t do any good. I’ll ask over on Dask — thanks!