Title
r

Riley Hun

04/14/2021, 10:51 PM
Hello, For the Prefect Server helm chart, I'm trying to expose the UI on an nginx controller ingress, but it's returning a 404 or 503 error. I can confirm that the UI was deployed successfully and it works when using a standard external load balancer. Concomitantly, I don't think the issue is nginx controller because I have exposed other applications on it. Also note that I have the UI working in tandem with nginx controller in a different environment (I think using a different version of the helm chart).
m

Michael Adkins

04/14/2021, 10:57 PM
Hey Riley, I'm not an expert on ingresses so I'm not sure where the best place to start is. Have you taken a look at the built-in ingress deployment though?
r

Riley Hun

04/14/2021, 10:57 PM
To replicate:
service:
    # type defines the service type and defaults to LoadBalancer
    # to expose the ui service to users and the UI.
    type: ClusterIP
    port: 8080
helm repo add ingress-nginx <https://kubernetes.github.io/ingress-nginx>
helm repo update
kubectl create ns nginx
helm install nginx ingress-nginx/ingress-nginx --namespace nginx --set rbac.create=true --set controller.publishService.enabled=true

kubectl apply -f prefect-ui-ingress.yaml
Hi Michael, Yes but would prefer to use nginx so I can enable basic-auth and also that's what I'm using in my other environment (which is working). I don't know why it's not working in this new environment though.
m

Michael Adkins

04/14/2021, 11:00 PM
We don't choose an ingress controller for you, you'd still need to use nginx.
Anyway, if it's working in one environment and not the other I'm afraid I won't be much help. Can you take a look at the browser webdev console and see if there are any more errors? (although 404/503 makes it unlikely)
r

Riley Hun

04/14/2021, 11:02 PM
It just says
site.webmanifest:1 Failed to load resource: the server responded with a status of 404 (Not Found)
m

Michael Adkins

04/14/2021, 11:02 PM
I've forwarded this thread to our devops team so they might be able to help out but it's after hours so we probably won't have more tips tonight.
r

Riley Hun

04/14/2021, 11:04 PM
Okay thank you! Appreciate it. There's no rush. I can try to figure things out on my side. The one working on the other environment is the helm chart before you went official (~1 year ago)
Any update on this issue? No luck on my side.
t

Tyler Wanner

04/16/2021, 10:04 PM
Hi Riley, generally this is outside the band of what we can support but I totally understand what you're trying to do here and I'd like to go through trying to set something like this up myself for a repro. If the UI is available via the load balancer service, it would seem everything is working--maybe a port or /path or annotation is misaligned somewhere. I'll try and get this nginx ingress controller setup going and let you know what I run into--seems pretty standard.
Hi Riley, I was able to get up and running with your setup and also experienced 503 and 404s until I made some changes. 503 would most likely be due to the basic-auth configuration with a missing secret for the auth password. Once it's properly authenticating, you will reach a 404 because of the /prefect-ui path. The UI should be reached on / not /prefect-ui, so you'll need to either reconfigure that or strip the URI from the request in nginx. Please let me know if that helps
unfortunately it looks like a rewrite will not solve this--you may need to put prefect server behind its own domain. instead of using your own custom ingress, you may use these values to create a proper ingress to create an ingress similar to the config you provided:
ui:
  service:
    # type defines the service type and defaults to LoadBalancer
    # to expose the ui service to users and the UI.
    type: ClusterIP
    port: 8080
  ingress:
    enabled: true
    path: /
    hosts:
      - <http://YOURDOMAIN.com|YOURDOMAIN.com>
    port: 8080
    annotations: 
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
      <http://nginx.ingress.kubernetes.io/auth-type|nginx.ingress.kubernetes.io/auth-type>: basic
      <http://nginx.ingress.kubernetes.io/auth-secret|nginx.ingress.kubernetes.io/auth-secret>: basic-auth
      <http://nginx.ingress.kubernetes.io/auth-realm|nginx.ingress.kubernetes.io/auth-realm>: 'Authentication Required - admin'
I can confirm that this works at YOURDOMAIN.com/ as long as there is a secret called basic-auth (followed instructions here)
e

Eric C

04/19/2021, 4:09 PM
Hi Tyler, I work with Riley - We tried changing the path to / and now it's working fine with the basic-auth. Thanks for your help!
🚀 1