https://prefect.io logo
Title
j

Juan David Lozano

02/11/2023, 8:21 PM
Hi I am following this tutorial to install prefect in google cloud https://medium.com/the-prefect-blog/prefect-server-101-deploying-to-google-cloud-platform-47354b16afe2, but when I get to the step
cd prefect && python3 -m pip install .
I get an error that prefect needs python 3.7, I try to upgrade to python 3.7 but I think (the repo was made with a lower version of python? I could be wrong just starting to understand docker) is there a more updated version of this article? or somebody can point me on how to install prefect on google cloud?
1
r

Ryan Peden

02/12/2023, 1:50 PM
Hi Juan! That tutorial shows how to set up Prefect 1. Unless you specifically wanted that, I recommend using Prefect 2. The good news is that setting up a Prefect 2 server is easier than Prefect 1. You have a couple of options: running Prefect using Docker/Docker Compose on a GCP VM, or running Prefect directly on a GCP VM. If you're familiar with Docker and want to run Prefect that way, I've created a GitHub repo that walks you the steps to get started. • Start a GCP virtual machine. I recommend picking an image that comes with a newer version of Python. Good choices might be: ◦ Rocky Linux 9 (if you prefer a distro based on Fedora and RHEL/CentOS) ◦ Ubuntu 22.10 ◦ Debian 11 • If you need or want to use a distro that comes with an old Python version, I recommend installing Miniconda and then using it to install a newer Python version. • If desired, create and activate a virtual environment using
venv
or
conda
. ◦ If Prefect is the only Python application you plan to run on this server, you can get by without a venv. I like to use them regardless to keep things neatly contained, but it's not required. • Install Prefect by running
pip install prefect
. This should install the newest version of Prefect (2.8.0). ◦ If/when you need to install a newer release of Prefect, you can do so by running
pip install -U prefect
. • Run
prefect orion start
to start up Prefect Server. ◦ By default it runs at
<http://127.0.0.1:4200>
. ◦ You can make it accessible from other machines, but before doing so, keeping mind this means that anyone who can reach the VM you're running Prefect on will be able to load your Prefect UI. If you're running on a private network, this is probably not a problem. Avoid running Prefect Server on an internet-visible VM unless you're sure you want to. To do it, you'll need to do one of: ▪︎ Run
prefect orion start --host 0.0.0.0
▪︎ Run
prefect config set PREFECT_ORION_API_HOST=0.0.0.0
▪︎ Then, regardless of which way you set the host, run
PREFECT_UI_URL=<https://my-awesome-server:4200/api>
• Note: update the UI URL to match the external hostname/IP of the VM. It should match whatever you'd enter in a web browser to load the Prefect UI, with
/api
added to the end. So if your server is running at
<https://my-awesome-server:4200>
, you'd set
PREFECT_UI_URL
to
PREFECT_UI_URL=<https://my-awesome-server:4200/api>
like I showed above. • That's it! Your Prefect server is set up and ready to use. • If you want it to run Prefect Server automatically when the VM starts, and automatically restart if it crashes, I recommend running Prefect Server as a Systemd service. ◦ I wrote a tutorial on how to do this for a Prefect agent, and doing it for Prefect server would be nearly identical. You can follow the instructions in that tutorial and just set your Prefect Server from above settings instead of the agent settings in the tutorial. Then, you can replace the
ExecStart
command with
prefect orion start
(or
prefect orion start --host 0.0.0.0
if you used that instead.)
That ended up being longer than I had planned 😆 Hopefully it helps you do what you need to do, though. I think I'll turn this into a blog or Discourse post so others can find and use it 🙂
j

Juan David Lozano

02/12/2023, 9:02 PM
@Ryan Peden thanks a million for the detailed instructions!!
r

Ryan Peden

02/12/2023, 9:20 PM
You're welcome! Here's the link to the Docker Compose example if you wanted to try it: https://github.com/rpeden/prefect-docker-compose I just realized I mentioned it but never actually linked to it.
j

Juan David Lozano

02/13/2023, 12:14 AM
@Ryan Peden so I followed your instructions, but I am getting an error when I run this command
prefect orion start --host 0.0.0.0
, in the end I am trying to connect to the prefect UI in that VM that you showed me how to create it, the error I get is this
I am attaching my network details in case tha helps
image.png
and sorry again very new at this , learning by trial and error
r

Ryan Peden

02/13/2023, 12:23 AM
Is there another copy of the Prefect server running? Looking back at my instructions, they might not be 100% clear. You need to either run
prefect orion start
or
prefect orion start --host 0.0.0.0
. If you run it on with the default setting (127.0.0.1:4200) and then try to run with the host arg without shutting down the other copy of the server, you'll get the error message you saw because port 4200 is already in use. If that's not what happened that I'm not sure what's wrong, but we will figure it out. 🙂
j

Juan David Lozano

02/13/2023, 12:35 AM
oh yes I first run the
prefect orion start
and then I ran the other line
prefect orion start --host 0.0.0.0
, is there a command to shut down the servers?
r

Ryan Peden

02/13/2023, 12:36 AM
You could also check if anything else is running on port 4200 by running
lsof -t -i:4200