Hi there, I am trying to get more information about how a Kubernetes Agent in prefect cloud is configured. The setup was done by a colleague and is working fine, but I have a hard time to understand which clusters/workloads within the Kubernetes Engine on GCP are actually used by the prefect agent. The config tab in the web UI is rather empty 🙂 Thanks for your help!

So for using the KubernetesAgent in production, the recommendation is to have it inside the cluster, and then it will use that cluster. If you have something outside of the cluster, I think it’s the equivalent of whatever

kubectl

is configured to.

@Kevin Kho thanks for your answer! Let me specify my question. I wasn't involved in the initial setup and therefore try to understand how the different parts work togehter. When I register a flow with

flow.run_config = KubernetesRun()

everything is working fine and the flow is using the already available KubernetesAgent. However, I only assume that this agent has a connection to a cluster in our GCP account and uses some workloads there. Probably prefect-agent in the screenshot triggers a job for every flow. But where is this connection between the prefect cloud KubernetesAgent and the GCP cluster/workload defined? And how can I be sure that the dask-workloads for example are not in use? Thanks!

The agent polls Prefect Cloud/Prefect Server every 10 seconds. It’s a one way connection and it looks for Flows to run. If it finds something to run, the agent is responsible for executing it. This is how we never see your code or data because you control all execution. If you have a Dask executor, the Flow will spin up the cluster (if you pass it a callable). I would not be able to know if your cluster is or is not in use. Because some people also have a long running cluster that they just connect to run their Flows

@Kevin Kho thanks, that makes it clearer! Hopefully the last question: how can I see what the GCP agent is actually polling in Prefect Cloud, i.e. which account and which agent. In the KUBECTL I can see values for • PREFECT__CLOUD__AGENT__AUTH_TOKEN • PREFECT__CLOUD__API (https://api.prefect.io) • PREFECT__CLOUD__AGENT__LABELS ('[]') • PREFECT__CLOUD__AGENT__AGENT_ADDRESS (http//8080) Is the token used to login to the right account and the labels define which agent in Prefect Cloud is polled? Thanks again!

I believe AUTH_TOKEN is told and it should be API_KEY, but yes this is the one that is attached to an account so 1 agent polls for the Flows in 1 tenant