https://prefect.io logo
Title
r

Rei Mendel

05/30/2022, 12:33 PM
Hey, 1. What is the best practice for running Tasks (not flows) over Kubernetes ? I think about using DaskExecuter and deploying Dask over Kubernetes cluster. Just want to make sure if that is the correct way 🙂 2. What is the best practice for managing CI/CD of flows? I wish to deploy and run branch specific flows and delete them upon merge Thanks!
a

Anna Geller

05/30/2022, 12:51 PM
are you asking for Prefect 1.0 or Prefect 2.0?
if you are just getting started, check out Prefect 2.0 - this thread gives more info
in that case you would use DaskTaskRunner and you can point it to any Dask cluster
there are plenty of CI/CD tools - many use GitHub actions for that - check this Discourse tag for more resources on the subject
in Prefect 1.0 you would use DaskExecutor and you could point it to an external Dask cluster or use it with KubeCluster as shown in those resources
r

Rei Mendel

05/30/2022, 1:47 PM
I'm using prefect 1. Thanks! I'll take a look. 🙂 much appreciated your quick and informative response
a

Anna Geller

05/30/2022, 2:17 PM
Sure 👍