Serdar Tumgoren
08/27/2021, 6:52 PMKevin Kho
Wilson Bilkovich
08/27/2021, 6:57 PMSerdar Tumgoren
08/27/2021, 6:59 PMWilson Bilkovich
08/27/2021, 7:00 PMdask-worker
label, and those are all deleted along with the scheduler + cluster.Wilson Bilkovich
08/27/2021, 7:01 PMkubectl logs
won’t help youSerdar Tumgoren
08/27/2021, 7:02 PMKevin Kho
Kevin Kho
Serdar Tumgoren
08/27/2021, 7:03 PMEvan Curtin
08/27/2021, 7:06 PMSerdar Tumgoren
08/27/2021, 7:07 PMEvan Curtin
08/27/2021, 7:07 PMSerdar Tumgoren
08/27/2021, 7:07 PMEvan Curtin
08/27/2021, 7:08 PMSerdar Tumgoren
08/27/2021, 7:11 PMEvan Curtin
08/27/2021, 7:12 PMSerdar Tumgoren
08/27/2021, 7:13 PMKevin Kho
Wilson Bilkovich
08/27/2021, 7:33 PMEvan Curtin
08/27/2021, 7:52 PMEvan Curtin
08/27/2021, 7:52 PMSerdar Tumgoren
08/27/2021, 7:55 PMSerdar Tumgoren
08/27/2021, 7:56 PMWilson Bilkovich
08/27/2021, 7:57 PMWilson Bilkovich
08/27/2021, 7:57 PMcron
expression to describe when you want it to be upSerdar Tumgoren
08/27/2021, 7:57 PMWilson Bilkovich
08/27/2021, 7:58 PMWilson Bilkovich
08/27/2021, 7:59 PMSerdar Tumgoren
08/27/2021, 7:59 PMWilson Bilkovich
08/27/2021, 8:02 PMWilson Bilkovich
08/27/2021, 8:03 PMWe'll use a node selector in our pods to make sure they run in the burst node pool. For this, we'll add a label to the nodes in that pool.
Serdar Tumgoren
08/27/2021, 9:08 PMSerdar Tumgoren
08/27/2021, 9:27 PMSerdar Tumgoren
08/27/2021, 9:27 PMWilson Bilkovich
08/27/2021, 9:38 PMWilson Bilkovich
08/27/2021, 9:39 PMWilson Bilkovich
08/27/2021, 9:40 PMpod_spec = make_pod_spec(
image="daskdev/dask:latest",
memory_limit="4G",
memory_request="4G",
cpu_limit=1,
cpu_request=1,
env={"EXTRA_PIP_PACKAGES": "prefect dask-kubernetes"},
)
executor = DaskExecutor(
cluster_class=KubeCluster,
cluster_kwargs=dict(
namespace="prefect",
pod_template=pod_spec,
n_workers=1,
),
)
Serdar Tumgoren
08/27/2021, 9:41 PMmake_pod_spec
or roll something custom if I’m unable to configure using that function. Either way, sounds like I need to make sure some of those other relevant pod configs are present. Thanks!Wilson Bilkovich
08/27/2021, 9:41 PMmake_pod_spec
makes it easy to set the other labels etcWilson Bilkovich
08/27/2021, 9:42 PMWilson Bilkovich
08/27/2021, 9:43 PMSerdar Tumgoren
08/27/2021, 9:43 PMWilson Bilkovich
08/27/2021, 9:47 PMSerdar Tumgoren
08/27/2021, 9:47 PMWilson Bilkovich
08/27/2021, 9:47 PM