Are there some public examples where people have t...
# ask-community
n
Are there some public examples where people have tried to pair Prefect with some HPC resources like clusters, schedulers, MPI, etc.. ? I am aware there is some level of support for Dask, but it is not completely obvious to me how to deploy it on a cluster. Suppose I can create/setup a Dask cluster, is it enough to provide
DaskTaskRunner(address="<http://my-dask-cluster>")
to the flow decorators? How do I control the type of the workers (CPU, GPU, etc..)? Is there MPI support through the Dask-MPI interface? More generally, what kind of limitations do I have compared to a conventional scheduler interface?