https://prefect.io logo
Title
m

Mike Lev

10/31/2021, 2:02 PM
hey all I am trying to deploy flow runs with a local agent on a local server started with
prefect server start
any full examples of such use cases or best practices? just for additional context running into some issues while working inside of a devcontainer where access to
localhost
is klunky if possible is there a sample flow file that shows how to run and deploy the flow on the server backend?
a

Anna Geller

10/31/2021, 3:13 PM
@Mike Lev deploying to Server shouldn’t differ in comparison to Prefect Cloud - in both scenarios, when you register your flow, it gets registered to the backend. The difference is only that Server is a single instance, by default running locally using docker-compose, while Prefect Cloud API and scheduler service are distributed and highly available. So there are no specific Server flow examples, afaik. But when it comes to configuring and deploying Server, perhaps this can be helpful: https://docs.prefect.io/orchestration/server/deploy-local.html#ui-configuration Btw, did you know that Prefect Cloud has 10000 free task runs each month? It’s much easier to get started with Cloud than with Server.
m

Mike Lev

11/07/2021, 6:03 PM
hey thanks we have decided to go with prefect cloud due to incomplete docs we are now having trouble deploying a simple flow from your repo onto our eks cluster is there a simple A-Z tutorial on deploying a simple flow to k8s (preferably geared towards eks) would help a ton as we are demoing this to decision makers that will decide budget allocated to license of prefect cloud
we have an agent up and running in our cluster in a namespace -
prefect_namespace
@Carmen
a

Anna Geller

11/08/2021, 9:00 AM
@Mike Lev
we are now having trouble deploying a simple flow from your repo
which flow and repo do you mean? do you mean those examples? When it comes to AWS EKS, afaik there are at least 2 tutorials but those were written some time ago so they might not be entirely up-to-date: this and this. However, the
KubernetesAgent
setup is quite similar for all cloud providers: once you installed prefect, authenticated your terminal and switched the context to your remote cluster (you can check with: kubectl config current-context), you can run:
prefect agent kubernetes install -k API_KEY
This will output a YAML declaration that you can adjust to your needs (e.g. adding label to the agent, adding any env variables you might need) and then apply to your Prefect namespace:
kubectl apply -f yourfile.yaml --namespace prefect_namespace
And once you have your agent running, you need to configure
KubernetesRun
run configuration to ensure that your flows will be deployed to exactly this agent’s cluster, here is one example for AWS: https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows/github_kubernetes_run_custom_ecr_image.py Sending some docs that may be helpful in case you didn’t see those yet: • https://docs.prefect.io/orchestration/agents/kubernetes.html#running-in-clusterhttps://docs.prefect.io/orchestration/flow_config/run_configs.html#kubernetesrunhttps://docs.prefect.io/api/latest/run_configs.html#kubernetesrun
m

Mike Lev

11/08/2021, 12:41 PM
phenomenal…. got it working thanks a ton @Anna Geller ! our rbac was jacked up
🙌 1