dh
06/30/2020, 7:50 PMZachary Hughes
06/30/2020, 8:23 PMdh
06/30/2020, 8:37 PMJoe Schmid
07/01/2020, 2:50 PMdh
07/01/2020, 4:26 PMJoe Schmid
07/01/2020, 4:29 PMdh
07/02/2020, 2:13 PMJoe Schmid
07/02/2020, 2:17 PMRegarding setting up the k8s stuff, I must say it has not been a smooth journey. 🙂I'm right there with you. Now that we're up and running it's great, but it wasn't the easiest process getting to this point.
Our on-prem HPC resources are managed by SLURM which Dask seems to have an adapter for [1]Yes, Dask has very good support for running on HPCs and my sense is that SLURM is one of the more popular resource managers. While I don't have direct experience of running Prefect & Dask on an HPC, I suspect you'll have a good experience, with good docs & a large community of Dask users on HPCs.
dh
07/02/2020, 7:34 PM