Michael C. Grant
07/22/2020, 3:49 PMJim Crist-Harif
07/22/2020, 3:50 PMMichael C. Grant
07/22/2020, 4:03 PMextraConfig
approach in the dask-gateway docs to provide an option handler, but it looks like extraConfig
is by default a dict, so the example in the docs needs some modification (will file an issue). But even when I fix that, even the stock dask-gateway container chokes on it.
Again, I know that's not enough context to really diagnose anything, and will follow up on more appropriate channels, but I figured that if someone already had a dask gateway worker with prefect installed it would be a great way to get started.Jim Crist-Harif
07/22/2020, 4:05 PMI tried to implement theIt can be either a dict or a str, both should work fine. If not, there's a bug in the helm chart.approach in the dask-gateway docs to provide an option handler, but it looks likeextraConfig
is by default a dict, so the example in the docs needs some modification (will file an issue).extraConfig
Michael C. Grant
07/22/2020, 4:07 PMIt can be either a dict or a str, both should work fine. If not, there's a bug in the helm chart.Helm seems to disagree, actually. At least later versions. It doesn't allow overriding a dict with a str
Jim Crist-Harif
07/22/2020, 4:09 PMMichael C. Grant
07/22/2020, 4:09 PMextraConfig:
config: |
from dask_gateway_server.options import Options, Integer, Float, String
def option_handler(options):
return {
"worker_cores": options.worker_cores,
"worker_memory": "%fG" % options.worker_memory,
"image": options.image,
}
c.Backend.cluster_options = Options(
Integer("worker_cores", 2, min=1, max=4, label="Worker Cores"),
Float("worker_memory", 4, min=1, max=8, label="Worker Memory (GiB)"),
String("image", default="daskgateway/dask-gateway:latest", label="Image"),
handler=option_handler,
)
Jim Crist-Harif
07/22/2020, 4:09 PMMichael C. Grant
07/22/2020, 4:11 PMcoalesce.go:196: warning: cannot overwrite table with non table for extraConfig (map[])
This might be Helm 3, but you're going to want to support Helm 3, it's very nice not to have to worry about tillerJim Crist-Harif
07/22/2020, 4:12 PMMichael C. Grant
07/22/2020, 4:13 PMdaskgateway/dask-gateway:latest
is not, in fact, the latest; it's 7 months old vs. 0.7.1
, which is 3 months oldJim Crist-Harif
07/22/2020, 5:51 PMMichael C. Grant
07/22/2020, 5:52 PMJim Crist-Harif
07/24/2020, 2:06 PMextraConfig
is a dict), then later add an extraConfig
as a str (instead of a map), you'll get a warning that the old config couldn't be merged with the new config, but the new config is still applied.
$ helm version
version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}
Shaun Cutts
09/05/2020, 3:56 AMdaskgateway/dask-gateway
which includes prefect, that I could/should use for further customization? With your speedy development, would be nice to have just one quickly improving dependency, rather than two. 🙂Jim Crist-Harif
09/08/2020, 2:20 PM