https://prefect.io
Join Slack
Hey, 1. What is the best practice for running Tasks (not flows) over Kubernetes ? I think about usin...
# ask-community
r
Hey, 1. What is the best practice for running Tasks (not flows) over Kubernetes ? I think about using DaskExecuter and deploying Dask over Kubernetes cluster. Just want to make sure if that is the correct way 🙂 2. What is the best practice for managing CI/CD of flows? I wish to deploy and run branch specific flows and delete them upon merge Thanks!
a
are you asking for Prefect 1.0 or Prefect 2.0?
if you are just getting started, check out Prefect 2.0 - this thread gives more info
in that case you would use DaskTaskRunner and you can point it to any Dask cluster
there are plenty of CI/CD tools - many use GitHub actions for that - check this Discourse tag for more resources on the subject
in Prefect 1.0 you would use DaskExecutor and you could point it to an external Dask cluster or use it with KubeCluster as shown in those resources
r
I'm using prefect 1. Thanks! I'll take a look. 🙂 much appreciated your quick and informative response
a
Sure 👍
4 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by