@Marvin I am migrating from Prefect 1 to Prefect 2. Previously in Prefect 1.0 we had a Kubernetes hosted agent that spun up jobs using a yaml file in our GCP environment. How can we do this in Prefect 2.0 (how can I create a Kubernetes Job that references an image I have in GCP and can use Kubernetes secrets)?

Ah, the sweet sound of success. It appears we've found some useful information on migrating your Kubernetes setup from Prefect 1 to Prefect 2. In Prefect 2, you can use the

KubernetesJob

task, which requires

kubectl

to be available. You must configure remote storage as local storage is not supported for Kubernetes. The ephemeral Prefect API won't work with Docker and Kubernetes, so you must have a Prefect server or Prefect Cloud API endpoint set in your agent's configuration. The Prefect CLI command

prefect kubernetes manifest server

automatically generates a Kubernetes manifest with default settings for Prefect deployments. You can pipe this output to a file of your choice and edit as necessary. More details can be found in the Prefect documentation. For creating a Kubernetes Secret for the API Key, you can follow the steps outlined in the Prefect Helm Worker guide. This guide also provides steps for adding the Prefect Helm repository to your Helm client, creating a namespace for the Prefect worker, and installing the Prefect worker using the Helm chart. Remember, the devil is in the details. Make sure to read through the documentation thoroughly to avoid any unexpected surprises. After all, we wouldn't want your migration to be more of a headache than it needs to be, would we?

hey @isabel obregon - marvin's a little behind here high level overview: • create a kubernetes type work pool (check out the advanced tab for any config you want to hardcode / override from your cluster env) • run a k8s worker with the helm chart • deploy flows that reference that work pool

thanks for the help @Nate! one thing we are still confused on is how we can specify where the kubernetes cluster is being hosted - we run our clusters on GCP, and are wondering how we communicate that to prefect in this new architecture

ahh, I would just follow the worker helm chart guide linked above so you just switch in the CLI to your GKE kubectl context (here ill just use my

docker-desktop

❯ k config get-contexts
  all
  your
  k8s
  contexts
  e.g.
  docker-desktop

❯ k config set-context --current --namespace=test
Context "docker-desktop" modified.

and now when you do all the

helm

commands here, the installation will take place in this context

thanks so much! taking a look through all of this now

sure! lmk if you have questions

Hi Nate! Working with Isabel on this - we really appreciate your help! We were able to get the worker deployed successfully. I think some last pieces that is still missing from our understanding is 1. secrets 2. source code Our flows are executing shell tasks to launch some typescript scripts. So we need the jobs to have access to our image. Is that something we configure in the worker or with some other means like a storage block? Same general question for secrets - we have some environment variables that the scripts need, do we specify these env variables on the worker level? And if so can we use kubernetes secrets and add them to the values.yaml?

1. secrets can be handled in many different ways: a. if you're comfortable doing so, you can use

prefect.blocks.system.Secret

(e.g. like this or in UI) to save secrets in cloud and access them wherever via the block SDK (i.e.

my_block = Secret.load("my-block-name")

) b. you can put environment variables as JSON in the

env

field of your work pool, and you can override that

env

on a deployment-basis if you desire (first pic) c. you can also avoid specifying any secrets in prefect, and define

pull

step in your

prefect.yaml

that modifies the

env

of the pod created by worker for a given flow run (docs) - for example, you could define a

run_shell_script

step in your

pull

section that fetches secrets from an external keyvault for whatever purpose you need 2. source code is an easier story a. if you want to keep code in a git-like env (my personal recommendation bc its going to be there anyways!) then you just define a

pull

step like this that will

git_clone

your flow code to your pod at runtime b. if you're going to use GCS or some other blob storage for code storage, you'd need to define a

push

and

pull

step like this but for GCS

Thank you for such an in depth response! Really appreciate it 😄

sure! lmk if any other questions arise