Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey,
1. What is the best practice for running Tasks (not flows) over Kubernetes ? I think about using DaskExecuter and deploying Dask over Kubernetes cluster. Just want to make sure if that is the correct way :slightly_smiling_face:
2. What is the best practice for managing CI/CD of flows? I wish to deploy and run branch specific flows and delete them upon merge
Thanks!

are you asking for Prefect 1.0 or Prefect 2.0?

if you are just getting started, check out Prefect 2.0 - <https://discourse.prefect.io/t/should-i-start-with-prefect-2-0-orion-skipping-prefect-1-0/544|this thread> gives more info

in that case you would use DaskTaskRunner and you can point it to any Dask cluster

there are plenty of CI/CD tools - many use GitHub actions for that - check <https://discourse.prefect.io/tag/ci-cd|this Discourse tag> for more resources on the subject

in Prefect 1.0 you would use DaskExecutor and you could point it to an external Dask cluster or use it with KubeCluster as shown in <https://discourse.prefect.io/tag/kube-cluster|those resources>

I'm using prefect 1. Thanks! I'll take a look. :slightly_smiling_face: much appreciated your quick and informative response