https://prefect.io logo
o

Oleg Sheyner

07/29/2022, 5:29 PM
Hi everyone, evaluating Prefect and hoping to get your advice on getting to a working cloud POC as fast as possible. Our situation: • Mostly Windows/Azure shop. Preference to run on Windows, all else equal, but can work with Linux VMs if necessary. • We have nothing in our ecosystem yet running on VMs (serverless only) and no Docker/Kubernetes setup. How would you suggest we set up a POC that deploys and runs flows on Azure and connects to Prefect Cloud: • POC should include demo for publishing and running flows to Azure from local dev machines • Primary goal is to set up as fast as possible: • Set up on a bare VM? Install and run under Docker? Try Azure AKS? (no experience at present with Kubernetes/AKS, but can be good case to acquire such experience in-house) Thanks much in advance!
👀 4
m

Mason Menges

07/29/2022, 6:59 PM
Hey @Oleg Sheyner Not super familiar with Azure But I'd start by checking out these docs they should help get you in the right direction. https://orion-docs.prefect.io/concepts/storage/ https://docs.prefect.io/concepts/deployments/ https://docs.prefect.io/concepts/infrastructure/
c

Chris Reuter

07/29/2022, 7:15 PM
@Christopher Boyd is our resident Azure pro and may have some advice
c

Christopher Boyd

07/29/2022, 7:49 PM
Hey @Oleg Sheyner - I think setting up through AKS isn’t too much trouble, and I have some public recipes in prefecthq/prefect-recipes to deploy an AKS cluster and the prefect agent / environment. With that said, it’s not up to date with GA and I need to update it. I have some additional content that I’m working on for deploying the server / agent to Kubernetes. For the most part, it’s pretty straightforward to get a basic setup through running the prefect agent, generating a manifest, and applying it via kubectl apply -f https://github.com/PrefectHQ/prefect-recipes/tree/main/azure/prefect-2/prefect-agent-on-aks https://discourse.prefect.io/t/prefect-1-0-prefect-2-0-kubernetes-run-config-to-kubernetesjob-infrastructure-block/1229
Notably, the prefect recipe I linked, you can use as is for the terraform aspect just to deploy your infrastructure. The prefect steps themselves (steps 6 and on in the readme) I need to go back over and confirm they are still accurate, but our documentation should be sufficient to apply the manifest from there
At a high level it would involve: Deploying your cluster Configuring your storage; if using 2.0, unfortunately we don’t have Azure blocks available just yet but are in progress. Starting your agent - this can be run either locally, or in-cluster. Local is simple to setup; in-cluster I have a document that is a work in progress , but there is an example manifest I shared yesterday to get it running. Creating your deployment Running your deployment
o

Oleg Sheyner

07/30/2022, 3:17 AM
@Christopher Boyd Sounds great, will review. In the meantime, to cover all possible avenues - what about deploying an image on a single Windows Server 2019 VM with Docker? Have people tried this, any writeups?
2 Views