Hi, i have kind of silly question, if i want to run agent, flow and server on same gke cluster can i have local agent instead of kubernetes agent?

I wouldn’t recommend it as you will likely face issues when you need to scale up or redeploy some components. For Server deployment on Kubernetes, I would recommend the helm chart. But @Aqib Fayyaz I remember we went together through both: • setting up a

KubernetesAgent

on GKE • as well as setting up Server with Helm on GKE and I remember we managed to do it (both), right? did something happen with your setup and you have to start from scratch?

yeah @Anna Geller i remember all of that worked but now its my job requirements to have docker-compose set up and working and than same way deploy it on gke as it works on docker-compose

Again, I wouldn’t recommend that since even if this works, you will 100% face issues with scale - I recommend using either Prefect Cloud or when you want to self host, then Helm Chart is the recommended setting for Kubernetes deployments. If you have very small workloads that fit into a single machine, you can deploy a single VM and self host Server using docker-compose. Docker-compose is meant for a single machine container deployments, not for something to be run on a Kubernetes cluster.

ok got it, but the thing is half of it has been deployed on gke our prefect code that runs the pipeline is already deployed on gke and it works now i only need to deploy server using this approch so that i can run the deployed pipeline when we want not that it should run automatically when deployed for the first time.

My recommendation is as follows: • if you want to use docker-compose, deploy your Server on a single VM, not GKE • if you want to deploy Server on a GKE Kubernetes cluster, use helm chart, not docker-compose.

Hi @Anna Geller so now i am using helm for server deployment and i have deployed it on gke following this https://github.com/PrefectHQ/server/tree/master/helm/prefect-server and this awesome video .

You can store both your flow and results in GCS (mounting cloud block storage volumes is more involved and I wouldn't do it unless you're a Kubernetes pro, especially given that you need it for object storage and GCS is made for that) as long as you have a service account permissions in your cluster, your flow run pods should be able to interact with GCS

yes even now the server is up on gke using helm chart the only thing which is added is shared volumes and i need to access them in my flow

you would need a persistent volume to mount a drive - check out those docs: https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes#persistentvolumeclaims that's correct, you need either Prefect Cloud or Server backend to run flows on schedule on GKE

i have them already

afaik you can't use persistent volume as flow storage, but you probably may use it for results if you specify the path. Again, I would recommend using GCS rather than persistent volume since you need to just store objects (both flow storage and results) and GCS is object storage, while persistent volume is block storage and it's used more for stateful applications like database API backend

Exactly i don't want to use persistent volume for flow but for result and this is what i want to know where the flow should be stored so that it can access the persistent volume using path specified in flow for storing the results in persistent volume

You can decide where you store it, there are no restrictions from the Prefect side.

Can you please tell me how things work like when flow is stored on docker file on gcr and server and agent on gke and when we run the flow from server how things work i mean who gets the flow and where it is run

This thread provides a detailed explanation

can i place my flow on vm instance when server and agents are on gke?

You are using Kubernetes agent right? If so, Prefect deploys your flow runs as Kubernetes jobs and those jobs must be able to pull your flow. So your Kubernetes job template would be the right way to configure that. But you need to use one of the existing storage mechanisms. If you really don't wanna use GCS, you may try using Local storage and store the flow on this PV, but it's at your own risk, I would really recommend using GCS for that (it's more reliable, scalable and even cheaper)

I really appreciate the great recomendation but i have only option of pv as other services need to get the result of flow and they are already looking into pv for that

I was speaking of Storage, not results - for results you can use whatever stateful mechanism you want (provided it's configured properly)

hmm ok so i will store the flow on gcp than how can i access the pv in flow got any idea?

When you specify PV claim, you specify the mount path and you could use the same path for your flow results in theory. Check out this for more info But again, I would strongly encourage you to use GCS for that instead - your custom applications can use the same storage bucket and paths to retrieve the flow results in the same way you would do it with a PV, there is really no difference apart from the fact that GCS is significantly less complex and more reliable/better fit for your use case. Is that you or someone else I need to persuade to GCS? 🙂 And if you still need to use PV, do you have some DevOps in your team who can help you with that? This is hard to support via Slack. Not sure if you saw that but we do provide a paid support for such infrastructure issues

Exactly, if you want to run a flow on Server backend, you need to register (and probably also schedule) your flow. Prefect doesn't support running flows as a long running service.

hmm what if i use docker storage for my flow in this way my flow will be able to run both as service (so that i should be able to mount pvc) and also i should be able to register the flow with server?

I don't know what you try to do. Can you explain the problem you try to solve?

ok lets get it straight i just need to mount file store instance to my flow no matter whever my flow run it just be able to communicate with with filestore instance (that is used as shared volume for all other services that we have on gke)

To access a file from a pod, you don't need to run flow as a service, you need to mount the PV to the pod, as we discussed before. There are some ways to do it, I would ask your DevOps folks to help you set this up and we also provide professional services you can book for such infrastructure issues. From Prefect perspective, you can set it on your Kubernetes job template, that's all I know and can help with tbh

ok Thank you so much for great help.

Sure, here is one example with AWS ECR, but you can use a similar logic with GCP GCR

incase of aws account id what i need to use for gcp?

your GCR registry url

and this is the docker file https://github.com/anna-geller/packaging-prefect-flows/blob/master/Dockerfile

yup, correct 🙂