Stéphan Taljaard

    Stéphan Taljaard

    1 year ago
    Hi. I have a Linux VM hosted on GCP that I 'm using as a single node for running Prefect Server + a LocalAgent. I added a unit file for each to auto-start them. I SSH'ed into the VM, installed docker + docker-compose, then _pip install_ed prefect. However, prefect is now only available under my user (GCP SSH logs into your GCP user by default). (
    which prefect
    ==
    /home/staljaard/.local/bin/prefect
    ) How would you recommend installing/configuring my prefect CLI and _pip install_ed prefect (agent) to be available for all users SSH'ing into the VM?
    Kevin Kho

    Kevin Kho

    1 year ago
    Hi @Stéphan Taljaard, I think the general question here is how to get any python library available to all users on Linux. Based on this , the solutions seem a bit ugly. In general though, Prefect saves flows and results in the user directory that I feel you may run into inconsistencies if you try to share stuff. I think the recommendation would be to make a common user.
    Michael Adkins

    Michael Adkins

    1 year ago
    You can also configure the Prefect home directory to be anywhere though 🙂
    YD

    YD

    1 year ago
    Hi, I've installed Prefect on a VM and having some issues... I can see the dashboard, but have issues with the agents. running
    prefect agent docker start
    , I see
    INFO - agent | Waiting for flow runs...
    but I do not see any agent when going to
    <VM IP>:8080/?agents=
    • How did you run the agent in a way that it is shows in the
    <VM IP>:8080/?agents=
    ? • How did you setup the Prefect connection in python code running on different machines (other VMs) to connect to that specific instance? Thanks
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @YD, I assume you did
    prefect backend server
    already? And also configured your API endpoint to that VM? Were you able to register a flow and have it appear?
    YD

    YD

    1 year ago
    @Kevin Kho I did run
    prefect backend server
    and
    prefect server start --detach
    not sure where are the instruction on how to configure my API endpoint to that VM. can you please point me to them ? flow does not appear and neither that agent
    Kevin Kho

    Kevin Kho

    1 year ago
    Check this . You want this configuration
    apollo_url="http://<your-ip-address>:4200/graphql"
    And then you want to make sure the VM ports allow the traffic to come in
    YD

    YD

    1 year ago
    Current ports
    just to make sure... if I did not create
    ~/.prefect/config.toml
    it should not exist? I do not see it in my home directory does it matter if I use docker and added
    alias docker-compose="TMPDIR=${HOME}/tmp docker-compose"
    to my .bash_profile ?
    Kevin Kho

    Kevin Kho

    1 year ago
    Just confirming that you have server on the VM and you’re trying to to register flows spin up an agent on a separate machine that hits the API on the VM right? The
    config.toml
    will be on the agent side/registration machine side. You can create it if it doesn’t exist. Let me find a template. Are you using Docker for the agent?
    This is an example of the
    config.toml
    More info about configuration in case you haven’t seen it yet.
    YD

    YD

    1 year ago
    I have a server on a VM1. for testing: I am trying to run basic code on that VM. just like I did when I ran Prefect on my laptop. 2. for usage: I want to run python code that uses Prefect server, on different machine, another VM or my laptop When I ran it on my laptop. in the python code I had
    LocalAgent().start()
    , this started an agent that stayed on. On the VM, running this basic code does not appear to work. I tried starting the agent from the python code and from the command line, both with and without docker. it looks like it starting an agent, but it just waits for flow run, and does not show up in the dashboard, and does not persist
    for reference, the
    hello-world
    code I am trying is '''
    from prefect import task, Flow, Parameter
    from prefect.agent.local import LocalAgent
    
    
    @task(log_stdout=True)
    def say_hello(name):
        print("Hello, {}!".format(name))
    
    
    def main():
        with Flow("hello_world") as flow:
            name = Parameter('name')
            say_hello(name='world')
    
        flow.run()
        LocalAgent().start()
        flow.register(project_name="Test")
        print('done')
    
    
    if __name__ == "__main__":
        main()
    '''
    Kevin Kho

    Kevin Kho

    1 year ago
    I see ok. So a couple of things. First, you normally would split out the flow and the agent starting. So in this case:
    def main():
        with Flow("hello_world") as flow:
            name = Parameter('name')
            say_hello(name='world')
        flow.run()
        # flow.register(project_name="Test")
    flow.run()
    would be for local testing.
    flow.register()
    would be to register to Cloud/Server when ready. Then you can start your agent with another script:
    from prefect.agent.local import LocalAgent
    LocalAgent().start()
    This would then start your agent if you call it with
    python ____.py
    . The reason is that the previous setup spins up an agent every time you register. You want to separate these. An agent can run multiple flows. You can also do
    prefect local agent start
    with the command line instead of the script. Now there are two things to test. You said that the agent is not showing in the UI. If you do
    python _____.py
    to register the flow, does it show in the UI? If the registered flow does not show in the UI, and the agent is running but does not show in the UI, then it means the API endpoint that these are hitting is not the one you want. By default with server, it will hit
    localhost:4200
    . You want it to hit
    VM IP:4200
    instead. This is why you need the
    config.toml
    in your
    .prefect
    folder in the home directory of the machine the agent is running on to point to the right API endpoint. You file should look like
    backend = "server"
    
    [server]
    host = "<http://SERVER-VM-IP>"
    port = "4200"
    host_port = "4200"
    This will make the agent and
    flow.register()
    send requests to the right API.
    YD

    YD

    1 year ago
    I still doing something wrong I want to run the Agent on the Prefect VM server (is this OK ?) I created
    config.toml
    in
    ~/.prefect
    with the following content:
    [server]
    host = "<http://aws31-x01-my01>"
    port = "4200"
    host_port = "4200"
    
    [server.ui]
    apollo_url="<http://aws31-x01-my01:4200/graphql>"
    then I ran
    prefect backend server
    prefect server stop
    prefect server start -d
    but it looks like it ignores the config file
    it still thinks the ui is on localhost
    When I put
    <http://aws31-x01-my01:4200/graphql>
    in my browser I get
    going to
    <http://aws31-x01-my01:8080/>
    opens the server UI, but tries to connect to the
    <http://localhost:4200/graphql>
    Well... In the UI I went to the
    Getting started
    page in that page I entered the
    graphql
    IP address then clicked connect and this did the trick it is now connected to the correct IP and I was able to create a docker agent.
    Hi @Kevin Kho Need to bother you again... Not clear to me, or I did not find the correct docs regarding to the next step. Now that I have the Prefect Server running on a VM and a Docker agent runs on the same VM. How do I use that Prefect server from my laptop or from a different VM ? (My goal is to have one Prefect server that serves several projects running on different VMs, so that there will be one UI to see all the different projects workflows) How does the Prefect server on the VM will start processes on my laptop?
    Kevin Kho

    Kevin Kho

    1 year ago
    You need the agent on your laptop configured to hit the API endpoint on the VM. The agent checks for flows to run every 10 seconds. If it finds flows to run, it will create the process on the laptop (or container or somewhere else). The agent is responsible for creating the process. You need to configure the agent with the IP address similar to what you did previously
    YD

    YD

    1 year ago
    So to make sure I understand.. On every machine that I want to use the Prefect server I need to • install prefect • setup the config.toml with Prefect server ip • start agent (do I need to do it for each user on the VM ?) I should not run
    prefect backend server
    or
    prefect server start
    or install Docker, correct?
    Kevin Kho

    Kevin Kho

    1 year ago
    You should run
    prefect backend server
    because the default is to use the Prefect Cloud API end point. You should not do
    prefect server start
    . You would need to install docker if your flows depend on it
    YD

    YD

    1 year ago
    Seams to be working... Thanks... thought I do see this in the Agent page
    is this a problem, or can I just ignore it ?
    Kevin Kho

    Kevin Kho

    1 year ago
    This means the agent is off. Did you turn off the process?
    YD

    YD

    1 year ago
    No.. it appears to be running
    Kevin Kho

    Kevin Kho

    1 year ago
    I think you mean your server is running, but I’m saying that agent is not running? That’s why the flows are late here. There is not agent to pick them up.
    YD

    YD

    1 year ago
    OK, will run the following on my laptop
    % prefect agent docker start
    It was ok for a minute, but then I got the notice again
    also tried to start agent with
    --agent-address http://<Server IP>:8080
    it did not like that
    Kevin Kho

    Kevin Kho

    1 year ago
    Since it’s yellow, I assume this was able to communicate with your server for a bit. Did you turn off the process? It has to continue running
    YD

    YD

    1 year ago
    I did not turn off anything, as far as I am aware. not on my laptop and not on the Prefect server VM
    Kevin Kho

    Kevin Kho

    1 year ago
    Could you show be what the agent is saying? Do you get something like this?
    You can also do
    prefect agent ____ start log-level=DEBUG
    and you can see why it’s struggling to connect to the Server
    YD

    YD

    1 year ago
    I do notice that it did not create a new agent when I started it on my laptop. the agent in the dashboard is the one I started on the Prefect server VM is this expected ?
    I see the issue.. the agent does not run in the background, I need to keep the terminal open. there is no --detach option
    Kevin Kho

    Kevin Kho

    1 year ago
    Yeah there is none. You need to use something like supervisor to keep the process running. The agent is meant to be a long running process
    YD

    YD

    1 year ago
    OK... checking this out
    Kevin Kho

    Kevin Kho

    1 year ago
    Just realized your profile picture LOL
    By the way, no pressure at all but some users just don’t know this. Prefect Cloud gives 10000 free task runs every month and is more than enough to get started with so that you don’t have to go through the pain of managing server (but you’ve nearly got it setup anyway)
    YD

    YD

    1 year ago
    I am big fun of the hitchhiker 😃 I am aware... this is my next step... our company is strict on security, so first I want to see I can use it on prem. then I will try the cloud. BTW, when I use the cloud service, will I also need to use
    supervisor
    ?
    Kevin Kho

    Kevin Kho

    1 year ago
    Yes for the agents. That’s right
    YD

    YD

    1 year ago
    it looks like when using
    supervisor
    I must provide label, is this expected ?
    Kevin Kho

    Kevin Kho

    1 year ago
    Not expected, labels should be separate from supervisor. Supervisor should just be for running the process. You may need matching labels to run your flows though. Are you familiar with labels?
    YD

    YD

    1 year ago
    When I started agent without
    supervisor
    I did not have to provide a label, but when I use
    supervisor
    I had to add
    --label  <computer name>
    for it to run. and it created a new agent on the Prefect server
    If I do not use a label I get
    Kevin Kho

    Kevin Kho

    1 year ago
    Oh ok I see what you are saying now. Yes I guess you need to provide the label then. Seems like it doesn’t attach the default label. I would also encourage you to add labels with the multiple machine setup to that the right machines pick up the corresponding flows