https://prefect.io logo
Title
s

Stéphan Taljaard

05/25/2021, 10:58 AM
Hi. I have a Linux VM hosted on GCP that I 'm using as a single node for running Prefect Server + a LocalAgent. I added a unit file for each to auto-start them. I SSH'ed into the VM, installed docker + docker-compose, then _pip install_ed prefect. However, prefect is now only available under my user (GCP SSH logs into your GCP user by default). (
which prefect
==
/home/staljaard/.local/bin/prefect
) How would you recommend installing/configuring my prefect CLI and _pip install_ed prefect (agent) to be available for all users SSH'ing into the VM?
k

Kevin Kho

05/25/2021, 1:32 PM
Hi @Stéphan Taljaard, I think the general question here is how to get any python library available to all users on Linux. Based on this , the solutions seem a bit ugly. In general though, Prefect saves flows and results in the user directory that I feel you may run into inconsistencies if you try to share stuff. I think the recommendation would be to make a common user.
z

Zanie

05/25/2021, 2:15 PM
You can also configure the Prefect home directory to be anywhere though 🙂
👍 1
y

YD

06/28/2021, 11:40 PM
Hi, I've installed Prefect on a VM and having some issues... I can see the dashboard, but have issues with the agents. running
prefect agent docker start
, I see
INFO - agent | Waiting for flow runs...
but I do not see any agent when going to
<VM IP>:8080/?agents=
• How did you run the agent in a way that it is shows in the
<VM IP>:8080/?agents=
? • How did you setup the Prefect connection in python code running on different machines (other VMs) to connect to that specific instance? Thanks
k

Kevin Kho

06/29/2021, 1:23 PM
Hey @YD, I assume you did
prefect backend server
already? And also configured your API endpoint to that VM? Were you able to register a flow and have it appear?
y

YD

06/29/2021, 7:47 PM
@Kevin Kho I did run
prefect backend server
and
prefect server start --detach
not sure where are the instruction on how to configure my API endpoint to that VM. can you please point me to them ? flow does not appear and neither that agent
k

Kevin Kho

06/29/2021, 7:51 PM
Check this . You want this configuration
apollo_url="http://<your-ip-address>:4200/graphql"
And then you want to make sure the VM ports allow the traffic to come in
y

YD

06/29/2021, 9:11 PM
Current ports
just to make sure... if I did not create
~/.prefect/config.toml
it should not exist? I do not see it in my home directory does it matter if I use docker and added
alias docker-compose="TMPDIR=${HOME}/tmp docker-compose"
to my .bash_profile ?
k

Kevin Kho

06/29/2021, 9:22 PM
Just confirming that you have server on the VM and you’re trying to to register flows spin up an agent on a separate machine that hits the API on the VM right? The
config.toml
will be on the agent side/registration machine side. You can create it if it doesn’t exist. Let me find a template. Are you using Docker for the agent?
This is an example of the
config.toml
More info about configuration in case you haven’t seen it yet.
y

YD

06/29/2021, 9:37 PM
I have a server on a VM 1. for testing: I am trying to run basic code on that VM. just like I did when I ran Prefect on my laptop. 2. for usage: I want to run python code that uses Prefect server, on different machine, another VM or my laptop When I ran it on my laptop. in the python code I had
LocalAgent().start()
, this started an agent that stayed on. On the VM, running this basic code does not appear to work. I tried starting the agent from the python code and from the command line, both with and without docker. it looks like it starting an agent, but it just waits for flow run, and does not show up in the dashboard, and does not persist
for reference, the
hello-world
code I am trying is '''
from prefect import task, Flow, Parameter
from prefect.agent.local import LocalAgent


@task(log_stdout=True)
def say_hello(name):
    print("Hello, {}!".format(name))


def main():
    with Flow("hello_world") as flow:
        name = Parameter('name')
        say_hello(name='world')

    flow.run()
    LocalAgent().start()
    flow.register(project_name="Test")
    print('done')


if __name__ == "__main__":
    main()
'''
k

Kevin Kho

06/29/2021, 9:52 PM
I see ok. So a couple of things. First, you normally would split out the flow and the agent starting. So in this case:
def main():
    with Flow("hello_world") as flow:
        name = Parameter('name')
        say_hello(name='world')
    flow.run()
    # flow.register(project_name="Test")
flow.run()
would be for local testing.
flow.register()
would be to register to Cloud/Server when ready. Then you can start your agent with another script:
from prefect.agent.local import LocalAgent
LocalAgent().start()
This would then start your agent if you call it with
python ____.py
. The reason is that the previous setup spins up an agent every time you register. You want to separate these. An agent can run multiple flows. You can also do
prefect local agent start
with the command line instead of the script. Now there are two things to test. You said that the agent is not showing in the UI. If you do
python _____.py
to register the flow, does it show in the UI? If the registered flow does not show in the UI, and the agent is running but does not show in the UI, then it means the API endpoint that these are hitting is not the one you want. By default with server, it will hit
localhost:4200
. You want it to hit
VM IP:4200
instead. This is why you need the
config.toml
in your
.prefect
folder in the home directory of the machine the agent is running on to point to the right API endpoint. You file should look like
backend = "server"

[server]
host = "<http://SERVER-VM-IP>"
port = "4200"
host_port = "4200"
This will make the agent and
flow.register()
send requests to the right API.
y

YD

06/29/2021, 10:37 PM
I still doing something wrong I want to run the Agent on the Prefect VM server (is this OK ?) I created
config.toml
in
~/.prefect
with the following content:
[server]
host = "<http://aws31-x01-my01>"
port = "4200"
host_port = "4200"

[server.ui]
apollo_url="<http://aws31-x01-my01:4200/graphql>"
then I ran
prefect backend server
prefect server stop
prefect server start -d
but it looks like it ignores the config file
it still thinks the ui is on localhost
When I put
<http://aws31-x01-my01:4200/graphql>
in my browser I get
going to
<http://aws31-x01-my01:8080/>
opens the server UI, but tries to connect to the
<http://localhost:4200/graphql>
Well... In the UI I went to the
Getting started
page in that page I entered the
graphql
IP address then clicked connect and this did the trick it is now connected to the correct IP and I was able to create a docker agent.
Hi @Kevin Kho Need to bother you again... Not clear to me, or I did not find the correct docs regarding to the next step. Now that I have the Prefect Server running on a VM and a Docker agent runs on the same VM. How do I use that Prefect server from my laptop or from a different VM ? (My goal is to have one Prefect server that serves several projects running on different VMs, so that there will be one UI to see all the different projects workflows) How does the Prefect server on the VM will start processes on my laptop?
k

Kevin Kho

06/30/2021, 11:18 PM
You need the agent on your laptop configured to hit the API endpoint on the VM. The agent checks for flows to run every 10 seconds. If it finds flows to run, it will create the process on the laptop (or container or somewhere else). The agent is responsible for creating the process. You need to configure the agent with the IP address similar to what you did previously
y

YD

06/30/2021, 11:42 PM
So to make sure I understand.. On every machine that I want to use the Prefect server I need to • install prefect • setup the config.toml with Prefect server ip • start agent (do I need to do it for each user on the VM ?) I should not run
prefect backend server
or
prefect server start
or install Docker, correct?
k

Kevin Kho

07/01/2021, 1:46 AM
You should run
prefect backend server
because the default is to use the Prefect Cloud API end point. You should not do
prefect server start
. You would need to install docker if your flows depend on it
y

YD

07/01/2021, 7:19 PM
Seams to be working... Thanks... thought I do see this in the Agent page
is this a problem, or can I just ignore it ?
k

Kevin Kho

07/01/2021, 7:19 PM
This means the agent is off. Did you turn off the process?
y

YD

07/01/2021, 7:22 PM
No.. it appears to be running
k

Kevin Kho

07/01/2021, 7:24 PM
I think you mean your server is running, but I’m saying that agent is not running? That’s why the flows are late here. There is not agent to pick them up.
y

YD

07/01/2021, 7:29 PM
OK, will run the following on my laptop
% prefect agent docker start
It was ok for a minute, but then I got the notice again
also tried to start agent with
--agent-address http://<Server IP>:8080
it did not like that
k

Kevin Kho

07/01/2021, 7:30 PM
Since it’s yellow, I assume this was able to communicate with your server for a bit. Did you turn off the process? It has to continue running
y

YD

07/01/2021, 7:37 PM
I did not turn off anything, as far as I am aware. not on my laptop and not on the Prefect server VM
k

Kevin Kho

07/01/2021, 7:40 PM
Could you show be what the agent is saying? Do you get something like this?
You can also do
prefect agent ____ start log-level=DEBUG
and you can see why it’s struggling to connect to the Server
y

YD

07/01/2021, 7:45 PM
I do notice that it did not create a new agent when I started it on my laptop. the agent in the dashboard is the one I started on the Prefect server VM is this expected ?
I see the issue.. the agent does not run in the background, I need to keep the terminal open. there is no --detach option
k

Kevin Kho

07/01/2021, 7:50 PM
Yeah there is none. You need to use something like supervisor to keep the process running. The agent is meant to be a long running process
y

YD

07/01/2021, 7:54 PM
OK... checking this out
k

Kevin Kho

07/01/2021, 7:55 PM
Just realized your profile picture LOL
By the way, no pressure at all but some users just don’t know this. Prefect Cloud gives 10000 free task runs every month and is more than enough to get started with so that you don’t have to go through the pain of managing server (but you’ve nearly got it setup anyway)
y

YD

07/01/2021, 8:01 PM
I am big fun of the hitchhiker :) I am aware... this is my next step... our company is strict on security, so first I want to see I can use it on prem. then I will try the cloud. BTW, when I use the cloud service, will I also need to use
supervisor
?
k

Kevin Kho

07/01/2021, 8:02 PM
Yes for the agents. That’s right
y

YD

07/01/2021, 9:44 PM
it looks like when using
supervisor
I must provide label, is this expected ?
k

Kevin Kho

07/01/2021, 9:48 PM
Not expected, labels should be separate from supervisor. Supervisor should just be for running the process. You may need matching labels to run your flows though. Are you familiar with labels?
y

YD

07/01/2021, 9:55 PM
When I started agent without
supervisor
I did not have to provide a label, but when I use
supervisor
I had to add
--label  <computer name>
for it to run. and it created a new agent on the Prefect server
If I do not use a label I get
k

Kevin Kho

07/01/2021, 10:02 PM
Oh ok I see what you are saying now. Yes I guess you need to provide the label then. Seems like it doesn’t attach the default label. I would also encourage you to add labels with the multiple machine setup to that the right machines pick up the corresponding flows