This is a bit of a stretch I suppose, has anybody ...
# randome
Evan Curtin
02/04/2022, 3:13 PMThis is a bit of a stretch I suppose, has anybody run MPI parallel jobs in k8s as a prefect task?
Evan Curtin
02/04/2022, 3:14 PMperhaps a light wrapper over https://github.com/kubeflow/mpi-operator ?
a
Anna Geller
02/04/2022, 3:23 PM
One community user tried that and it doesn’t work because kubeflow uses custom Kubernetes job types which won’t work with Prefect Kubernetes agent.
Did you try mapping with Prefect? It allows you to spin up thousands of child tasks that can be parallelized e.g. across a KubeCluster - all with just Prefect and Dask. LMK if you need more info
k
Kevin Kho
02/04/2022, 3:26 PM
Are you using MPI because you have an HPC?
e
Evan Curtin
02/04/2022, 4:56 PMThe libraries we are working with are built around MPI
Evan Curtin
02/04/2022, 4:57 PMI’ve inherited a legacy codebase
Evan Curtin
02/04/2022, 4:58 PMI’m quite comfortable wrapping a CRD into a prefect task actually
k
Kevin Kho
02/04/2022, 4:58 PM
Ah I see
e
Evan Curtin
02/04/2022, 4:59 PMlike for example, if we wanted to do large scale SVD using https://slepc.upv.es/
k
Kevin Kho
02/04/2022, 5:01 PM
Holy cow you must have big data to be needing MPI for an SVD operation 😆. Is this for recommendation?
e
Evan Curtin
02/04/2022, 5:04 PMwe’re very early on investigating options, due to the complexity of that I don’t see us doing it practically but maybe itll be a fun investigation
11 Views