Are there instructions on how to set up Kubernetes flows in

Question

Are there instructions on how to set up Kubernetes flows in Prefect Cloud 2 0

Chris Hansen · Answer

i m aware of <https docs prefect io tutorials kubernetes flow runner |https docs prefect io tutorials kubernetes flow runner > but i don t need all of Prefect 2 0 on kubernetes

Taylor Curran · Answer

Hi Chris Have you checked out our new concept <https docs prefect io concepts infrastructure |infrastructure>

Taylor Curran · Answer

From the Docs Infrastructure is specific to the environments in which flows will run Prefect currently provides the following infrastructure types `Process` runs flows in a local subprocess `DockerContainer` runs flows in a Docker container `KubernetesJob` runs flows in a Kubernetes Job

Chris Hansen · Answer

i haven t but this is a great reference the part i m still trying to figure out what the right way to set up kubernetes agents how do i get the agent to listen to prefect cloud

Taylor Curran · Answer

Have you read up on our concept of <https docs prefect io concepts work queues |workqueues and agents>

Chris Hansen · Answer

how is a k8s pod that acts as an agent different from one that executes flows i see they both need prefect installed

Tom Klein · Answer

< Taylor Curran> the docs specifically the last link you sent that discusses agents are not so clear do we need to manually create a pod and a container that runs `prefect agent run` in prefect 1 0 prefect itself provided an easy way to do it ```prefect agent kubernetes install k API KEY | kubectl apply namespace=my namespace f ``` this is taken directly from the prefect 1 0 docs Have things regressed so that now users need to manually create amp deploy a container in k8s that would start the agent ourselves

Tom Klein · Answer

i m referring to this documentation of 1 0 just to clarify <https docs v1 prefect io orchestration agents kubernetes html custom job template>

Christopher Boyd · Answer

Christopher Boyd · Answer

Still in progress and video to follow but hopefully this should get you started

Christopher Boyd · Answer

for the agent specifically it s currently geared towards open source and not configured for cloud with the default manifest I m working on another discourse document that will walk through your exact concerns setting up the agent to cloud api

Christopher Boyd · Answer

I have a working manifest that can be used in 2 0 to deploy the manifest and point to the cloud api

Ilya Galperin · Answer

Thank you Christopher Is there somewhere we can find this manifest or a particular space we should be watching for it to be published soon

Tom Klein · Answer

< Christopher Boyd> thanks a few questions if you may and sorry for hijacking the thread is the current state of things the desired one or is it just relatively early in the release process and not everything is finalized in other words is the k8s agent supposed to be a native part of prefect 2 0 or is it basically becoming some niche solution for advanced users i thought about migrating to 2 0 but as it is right now i m not even sure what to ask my DevOps to do out of sheer curiosity what drove the decision to rely on `yaml` files rather than to rely on python config as code for example why shouldn t the different deployment s of each flow just be first class python objects that are registered the same way flows were registered in 1 0 e g via a simple python API is that something that s planned and just not ready yet or not planned at all can you provide an example for this manifest

Anna Geller · Answer

interesting question all decisions are driven by common user problems and we believe this new deployment approach provides a better UX this may help understand why we did it this way <https discourse prefect io t deployments are now simpler and declarative 1255>

Christopher Boyd · Answer

< Ilya Galperin> this is something I ve been working through on my own but will hopefully be able to contribute soon as published documentation in the official docs discourse I hate to say it s a work in progress but I m working on it < Tom Klein> that I can t speak for as I m not involved with or engaged in the product design That said we want to make all the things better Just in the amount of content and work that goes in though sometimes it takes some time to see what needs some extra scrutiny and I d love to make this easier for everyone to understand implement and use I can provide a current example but this is in no way official at the moment and just a working copy I have locally for the agent

Christopher Boyd · Answer

```apiVersion apps v1 kind Deployment metadata name cloud2 agent namespace orion spec selector matchLabels app prefect agent replicas 1 template metadata labels app prefect agent spec containers name agent image prefecthq prefect 2 0 0 python3 9 command prefect agent start kubernetes imagePullPolicy IfNotPresent env name PREFECT API URL value lt account id gt workspaces lt workspace id gt name PREFECT API KEY valueFrom secretKeyRef name prefectapi key PREFECT API KEY apiVersion kind Role metadata namespace orion name flow runner rules apiGroups resources pods pods log pods status verbs get watch list apiGroups batch resources jobs verbs get list watch create update patch delete apiVersion kind RoleBinding metadata name flow runner role binding namespace orion subjects kind ServiceAccount name default namespace orion roleRef kind Role name flow runner apiGroup ```

Christopher Boyd · Answer

Note specifically the API url will need to be updated for yours and a secret saved in your namespace

Tom Klein · Answer

thanks again < Christopher Boyd> and thanks < Anna Geller> for the link i actually have a Q about something that s written there but i don t wanna divert this thread further than i already did should i open a new one

Christopher Boyd · Answer

Which link are you referring to the one Anna shared or the one I shared

Tom Klein · Answer

the last one < Anna Geller> shared that explains deployments