Serdar Tumgoren
08/27/2021, 6:52 PMKevin Kho
08/27/2021, 6:55 PMWilson Bilkovich
08/27/2021, 6:57 PMSerdar Tumgoren
08/27/2021, 6:59 PMWilson Bilkovich
08/27/2021, 7:00 PMdask-worker
label, and those are all deleted along with the scheduler + cluster.kubectl logs
won’t help youSerdar Tumgoren
08/27/2021, 7:02 PMKevin Kho
08/27/2021, 7:02 PMSerdar Tumgoren
08/27/2021, 7:03 PMEvan Curtin
08/27/2021, 7:06 PMSerdar Tumgoren
08/27/2021, 7:07 PMEvan Curtin
08/27/2021, 7:07 PMSerdar Tumgoren
08/27/2021, 7:07 PMEvan Curtin
08/27/2021, 7:08 PMSerdar Tumgoren
08/27/2021, 7:11 PMEvan Curtin
08/27/2021, 7:12 PMSerdar Tumgoren
08/27/2021, 7:13 PMKevin Kho
08/27/2021, 7:32 PMWilson Bilkovich
08/27/2021, 7:33 PMEvan Curtin
08/27/2021, 7:52 PMSerdar Tumgoren
08/27/2021, 7:55 PMWilson Bilkovich
08/27/2021, 7:57 PMcron
expression to describe when you want it to be upSerdar Tumgoren
08/27/2021, 7:57 PMWilson Bilkovich
08/27/2021, 7:58 PMSerdar Tumgoren
08/27/2021, 7:59 PMWilson Bilkovich
08/27/2021, 8:02 PMWe'll use a node selector in our pods to make sure they run in the burst node pool. For this, we'll add a label to the nodes in that pool.
Serdar Tumgoren
08/27/2021, 9:08 PMWilson Bilkovich
08/27/2021, 9:38 PMpod_spec = make_pod_spec(
image="daskdev/dask:latest",
memory_limit="4G",
memory_request="4G",
cpu_limit=1,
cpu_request=1,
env={"EXTRA_PIP_PACKAGES": "prefect dask-kubernetes"},
)
executor = DaskExecutor(
cluster_class=KubeCluster,
cluster_kwargs=dict(
namespace="prefect",
pod_template=pod_spec,
n_workers=1,
),
)
Serdar Tumgoren
08/27/2021, 9:41 PMmake_pod_spec
or roll something custom if I’m unable to configure using that function. Either way, sounds like I need to make sure some of those other relevant pod configs are present. Thanks!Wilson Bilkovich
08/27/2021, 9:41 PMmake_pod_spec
makes it easy to set the other labels etcSerdar Tumgoren
08/27/2021, 9:43 PMWilson Bilkovich
08/27/2021, 9:47 PMSerdar Tumgoren
08/27/2021, 9:47 PMWilson Bilkovich
08/27/2021, 9:47 PM