Hello prefect! I’m an ML engineering working on computer vision for cancer diagnosis.
At our company we have an HPC and use Slurm as our scheduler. We are trying to integrate prefect into our pipeline and are currently working on writing a Slurm task executor, hoping this is all we’ll need to build too use prefect for our usecase.
If anyone has any resources or suggestions for us, they would be much appreciated, thanks!
PS: We cannot use Dask because of incompatibilities with PyTorch
12/02/2020, 4:07 PM
Welcome @Rodrigo Ceballos! Don’t have any specific suggestions for you, but this is an exciting idea 🙂