Will prefect 2.0 be supported via kubernetes GKE a...
# ask-communityj
James Phoenix
07/29/2022, 1:24 PMWill prefect 2.0 be supported via kubernetes GKE auto-pilot?
j
Joshua Greenhalgh
07/29/2022, 1:27 PMI hope soo! I can't imagine there would be anything that could mean it is not? What are you concerned about?
j
James Phoenix
07/29/2022, 1:29 PMI am new to prefect and just wanted to know if it is possible 😉
James Phoenix
07/29/2022, 1:29 PMI’ll try auto-pilot first I think 😛
j
Joshua Greenhalgh
07/29/2022, 1:38 PMI think in most cases they are interchangeable - the main diff is that google is managing the autoscaling - but the k8s apis are exactly(ish) the same I think
r
Rajvir Jhawar
07/29/2022, 2:08 PMI deploy on GKE autopilot and have no problems
j
James Phoenix
07/29/2022, 2:09 PMPerfect that’s good to know, thanks @Rajvir Jhawar
James Phoenix
07/29/2022, 2:09 PMWhen I deploy my flows into buckets, how do I attach additional dependencies? @Rajvir Jhawar?
r
Rajvir Jhawar
07/29/2022, 2:11 PM@James Phoenix The only issue you encounter with GKE autopilot is the cost associated with running an agent 24/7. GKE autopilot bills per hour your pod is running. The great thing about prefect is you can run you agent outside your cluster (you lose scaling + auto healing) but the price difference might be worth it
Rajvir Jhawar
07/29/2022, 2:12 PMI run my agent on a seperate VM (make sure you have kubectl installed on said VM and then connect it to your cluster)
Rajvir Jhawar
07/29/2022, 2:14 PM@James Phoenix i not quite sure i understand your question. Do you mean third party libraries?
Then you would package them up in a docker image and deploy to CR (container register).
There are others way of doing it, depends on your devops setup
j
James Phoenix
07/29/2022, 2:14 PMAh gotcha, so if I am running a VM it is cheaper than K8s for the deployment? @Rajvir Jhawar Perhaps I should use a standard k8s setup then with a lower number of resources? I only want this for my own personal usage.
James Phoenix
07/29/2022, 2:14 PMGotcha, so use docker images via Github container registry / Docker hub
r
Rajvir Jhawar
07/29/2022, 2:18 PMGKE autopilot is billed basically by how long you run a pod for:
So what i do to save some money is have my k8 jobs run in the cluster, but then have the agent in a separate VM constantly polling the prefect api then it would send the requests to your cluster.
That way your not paying for the price of an agent running 24/7.
Now you go the route of using a standard k8 cluster that also up to you.
I prefer autopilot for ease of setup. Also if it fits into your budget just deploy your agent there as well.
Rajvir Jhawar
07/29/2022, 2:19 PMYea whatever docker container register you want to use is going up to you. As long as prefect knows where your images are stored you are good
3 Views