https://prefect.io logo
m

Mia

03/29/2022, 12:43 PM
Hello, I’m new to Prefect and I need some help getting started. I’m currently struggling with two issues: 1. I’m setting up Prefect server on EKS using Helm. After
helm upgrade
ran successfully, I get commands such as
Copy code
1 Run the following command to get the UI URL:

  UI_HOST=$( \
    kubectl get svc \
    --namespace prefect \
    --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}" \
    prefect-server-ui \
  ) \
  && echo "UI available at: http://$UI_HOST:8080"...
when I run the above command, I get an error saying
Copy code
error: error executing template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}": template: output:1:10: executing "output" at <index .status.loadBalancer.ingress 0>: error calling index: index of untyped nil
Then I have to run
kubectl port-forward <name of the service> <port>:<port>
to get ui and apollo servers running. Is there a way to run this without port forwarding? 2. I’m trying to run a hello world example flow on prefect server on my EKS cluster. I tried to trigger it from my command line by creating a project and running
prefect register --project test-project -p hello/
. The registering seem to run successfully but I don’t see the flow on the ui. I’m also not able to manually create flow project using ui. Am I interacting with two different prefect engine here? How do I register prefect flow so that it is running on my prefect server on EKS cluster?
I was able to change
prefect backend server
so i think I’m no longer running prefect local. But I still have the issue running the commands to get the UI URL and Apollo URL
👍 1
a

Anna Geller

03/29/2022, 1:30 PM
Based on this, you should use ClusterIP instead of LoadBalancer
Regarding the registration issue, did you configure this in config.toml in the registration environment?
Copy code
[server]
endpoint = "<http://YOUR_MACHINES_PUBLIC_IP:4200/graphql>"

  [server.ui]
    apollo_url = "<http://YOUR_MACHINES_PUBLIC_IP:4200/graphql>"
doing this config.toml change + switching to server backend should fix the issue
m

Mia

03/29/2022, 1:35 PM
where is the config.toml? Should I include that in the project root folder?
a

Anna Geller

03/29/2022, 1:37 PM
To register flows to your Sever you need to install Prefect on your machine. This should create the config.toml.
m

Mia

03/29/2022, 1:57 PM
I’ve set up prefect using Helm Chart and followed the instructions here but this doesn’t seem to mention config.toml…
a

Anna Geller

03/29/2022, 2:01 PM
because this README is for setting up your orchestration layer. To start using it, check out those docs: https://docs.prefect.io/orchestration/
m

Mia

03/29/2022, 2:22 PM
what’s the difference updating the endpoint in values.yaml for Helm Prefect upgrade vs updating config.toml?
a

Anna Geller

03/29/2022, 2:31 PM
The same difference as before. Helm chart deploys Server as your orchestration layer. Your execution layer could be something entirely different - e.g. you could have an agent running on your local machine. The config.toml is for your execution layer - something that would be accessed by your agent and by your registration environment
m

Mia

03/29/2022, 6:46 PM
Thanks. I was able to get the UI and Apollo server running without altering config.toml but updating values in my values.yaml file for Helm
Is there by any chance an example how to register and run a prefect flow in kubernetes cluster? i have my hello world example ready but I’m not sure how to run this on my prefect server in kubernetes cluster
and the documentation is a bit confusing
a

Anna Geller

03/29/2022, 7:18 PM
what exactly is confusing? and why would you want to register from a Kubernetes pod? normally, most users register their flows either from their local machine or from CI
m

Mia

03/29/2022, 7:35 PM
So I have a Kubernetes cluster running. I was able to set up prefect with Helm Chart there. Now locally I wrote a hello_world.py flow script and have it ready. How do I get this hello_world.py to be deployed to the prefect server that’s in my Kubernetes cluster?
a

Anna Geller

03/29/2022, 8:58 PM
you would need to install prefect on your local machine, change backend to server (prefect backend server) and then add the Server endpoint (that gets printed when you deploy Server with Helm chart) to your config.toml available on your local machine in:
Copy code
~/.prefect/config.toml
you only have to add `YOUR_MACHINES_PUBLIC_IP`:
Copy code
[server]
endpoint = "<http://YOUR_MACHINES_PUBLIC_IP:4200/graphql>"
m

Mia

03/30/2022, 1:17 PM
Thanks! That’s what I was guessing so it’s good to have a confirmation:) Regarding the updating server endpoint in config.toml, does it have to be an public IP address? can I use dns record instead?
a

Anna Geller

03/30/2022, 1:48 PM
it must be reachable from wherever you register your flows. If your laptop is connected to the same VPN your Server is running on, this should work in theory
m

Mia

03/30/2022, 5:28 PM
Thank you. I was able to configure and run it successfully
a

Anna Geller

03/30/2022, 5:30 PM
nice work! 👏
125 Views