Folks Good evening afternoon morning slightly smiling face D

Question

Folks Good evening afternoon morning slightly smiling face Does anyone has OpenShift Yaml templates to set up PREFECT Due to privacy reason we cant use Prefect Cloud So need to set up everything on our private OpenShift Anyone has Openshift Build Deploy yaml templates

Kevin Kho · Answer

That we do not We just have the helm chart Can you use helm with your openshift

jawnsy · Answer

OpenShift should support Helm fine new versions of 4 x actually have a built in graphical wizard too which is really nice One caveat is that a lot of our images currently run as root so you need to make sure that you re using the anyuid security context constraint for your service account We haven t tested it but I used to be an engineer at Red Hat working on OpenShift and have some experience with it so I m happy to help you get things working

Kevin Kho · Answer

Here is the <https github com PrefectHQ server tree master helm prefect server|helm chart>

Abin Antony · Answer

Thanks < Kevin Kho> < jawnsy>

davzucky · Answer

All the perfect image are rootless now and run on openshift We are running this setup on our side

Abin Antony · Answer

I am trying to install Prefect on our private Openshift and getting an error like this What could be the cause Could you please help with your thoughts

jawnsy · Answer

I guess the error message is saying that you re not allowed to create a loadbalancer or nodeport service my guess is that you need to use a ClusterIP and then create a Route or Ingress

Abin Antony · Answer

< jawnsy> you mean to say I have less permission on OCP Project which I trying to install

jawnsy · Answer

you probably need to change this to `type ClusterIP` and then enable the ingress or manually create an ingress Recent versions of OpenShift 4 should understand Ingress and translate it into a Route but you might need some annotations to control the behavior annotations in the OpenShift doc

Abin Antony · Answer

I changed 2 instances of type LoadBalancer on values yaml to type ClusterIP and deployment went fine Thanks for that < jawnsy> But seems like not all PODs are up Are they any minimum resource requirement for Prefect Services I can see my memory is 100

Abin Antony · Answer

< jawnsy> We have an increased resource on our OCP now But after helm install seems like postgres POD is not up and I am seeing an error like this `create Pod prefect server 1656368390 postgresql 0 in StatefulSet prefect server 1656368390 postgresql failed error pods prefect server 1656368390 postgresql 0 is forbidden unable to validate against any security context constraint provider anyuid Forbidden not usable by user or serviceaccount provider pipelines scc Forbidden not usable by user or serviceaccount provider restricted spec securityContext fsGroup Invalid value int64 1001 1001 is not an allowed group spec containers 0 securityContext runAsUser Invalid value 1001 must be in the ranges 1011190000 1011199999 provider nonroot Forbidden not usable by user or serviceaccount provider rsync anyuid Forbidden not usable by user or serviceaccount provider hostmount anyuid Forbidden not usable by user or serviceaccount provider aqua scc Forbidden not usable by user or serviceaccount provider elasticsearch scc Forbidden not usable by user or serviceaccount provider log collector scc Forbidden not usable by user or serviceaccount provider machine api termination handler Forbidden not usable by user or serviceaccount provider hostnetwork Forbidden not usable by user or serviceaccount provider hostaccess Forbidden not usable by user or serviceaccount provider node exporter Forbidden not usable by user or serviceaccount provider privileged Forbidden not usable by user or serviceaccount provider trident Forbidden not usable by user or serviceaccount `

jawnsy · Answer

Did you add your service account to the anyuid security context constraint If not or if you can t do that you will need to modify the chart and we can t really help too much with that It looks like the error message is because OpenShift is not permitting our deployments to run as user 1001 as our Helm charts require If you want to run under the restricted scc then you might need to build your own images to ensure that files are readable by the restricted SCC user e g group readable

jawnsy · Answer

There s some advice here under Support arbitrary uids <https docs openshift com container platform 4 10 openshift images create images html>

Abin Antony · Answer

< jawnsy> Sorry I havent added any SA Could you please point me to list of SAs required for me to set up PREFECT I can give a try

jawnsy · Answer

The Helm chart should create service accounts but the service accounts will only be allowed to use the restricted scc by default and you d need to grant them anyuid or nonroot I think the service account is defined here You can try running `oc get sa` in your project namespace and see what comes up The docs about SCCs are here There s a command you have to run to add the nonroot scc to the service account

jawnsy · Answer

it s an oc adm command somewhere

jawnsy · Answer

Abin Antony · Answer

< jawnsy> I checked with our OCP administrator and they mentioned Only the restricted SCC is allowed anyuid or noonroot is NOT allowed So if I understand your previous statements correctly then only wayforward is to create our own docker images Right

Abin Antony · Answer

Because using PREFECT helm chart its trying to add 1001 user

jawnsy · Answer

Currently yes Either build your own images or you might be able to do it by customizing the Helm installer only I m not sure We also have a professional services team that can make sorts of changes

Abin Antony · Answer

< jawnsy> What all services need to have this elevated permissions 1001 user Like Hasura Apollo Towel UI etc

Abin Antony · Answer

Because I can see this requirement only for Hasura

jawnsy · Answer

I m not sure if it s a requirement per se the default behavior on non OpenShift installs of Kubernetes is to use the USER from the image the user of the last layer when the deployment manifest does not define one It may work properly with a different user but we have not tested Typical things that cause these containers to fail on OpenShift are file permissions as described in one of the articles I linked earlier in the OpenShift docs because if we build the image as USER 1001 and mark the file as 700 then we expect it to be readable by the running image since in most k8s distros it will run as user 1001 But in OpenShift we need group permissions or else the UID that the restricted SCC assigns will not be able to read the files since the container is running as the restricted SCC UID instead of the image layer UID

jawnsy · Answer

You can try to see if you can reproduce the problems by running with podman or docker and explicitly setting a different uid e g `docker run rm it u 123456789 0` OpenShift runs using the root group and a randomized per namespace UID

Abin Antony · Answer

< jawnsy> Well I am trying to get restrictions lifted on our OC project by our admin team so that it will be a smooth sail without any major customization on PREFECT services Curios to know < davzucky> How was your set up on OCP Does your platform allow anyuid non root users

Abin Antony · Answer

< jawnsy> I made progress and set up Prefect on OCP and all pods are up and running Now when I am trying add new project it s not working or adding projects to the system What could you be the cause

jawnsy · Answer

It might be a database connectivity problem Hard to say without logs

Abin Antony · Answer

Ok This is what I am seeing at the browser console logs Nothing at POD level

Abin Antony · Answer

This is the app <https prefectuiroute e11dbf dev apps klab devops gov bc ca > Could you please let me know which log you were mentioning

jawnsy · Answer

In the OpenShift console the logs for the various deployments

Abin Antony · Answer

No errors at POD container level I exposed Openshift routes only for UI and graphql APOLLO services is there any other service that is getting consumed from UI which i need to create public routes

Anna Geller · Answer

sorry Abin we ve shared everything we know about the problem and it s hard to troubleshoot such infrastructure issues via Slack if you need more guidance here I recommend you reach out to a paid support <mailto cs prefect io|cs prefect io>