Hi can someone point out what I m missing in the following s Prefect Community #ask-community

Hi - can someone point out what I'm missing in the...

Philip MacMenamin

08/10/2021, 1:57 PM

Hi - can someone point out what I'm missing in the following set up - I have a Server stood up, and separate machine running:

Copy code

prefect agent local start --api http://<server_IP>:4200

Philip MacMenamin

08/10/2021, 1:58 PM

I've registered the flows on the agent machine, and the server can see the agent as is evident in the GUI. However, the registered flows are not showing up.

Kevin Kho

08/10/2021, 2:04 PM

Hey @Philip MacMenamin, did you do

prefect backend server

Philip MacMenamin

08/10/2021, 2:05 PM

I did... on both machines actually now that I think about it 🤔

Kevin Kho

08/10/2021, 2:56 PM

Is the flow you registered from on a different machine than the server? How did you configure the API endpoint?

Philip MacMenamin

08/10/2021, 3:06 PM

Essentially I've been running the agent on the same machine as the server, and now I'd like to run the flows on another machine. All I've done is stop the agent that's running on the server machine, and started up an agent on another machine running a prefect agent, I registered the flows on the agent machine.

Philip MacMenamin

08/10/2021, 3:10 PM

Maybe there's a walkthrough on this that I'm missing. Conceptually I thought you could set up agents running on various machines, which can connect to a server (via 4200) and await jobs.

Philip MacMenamin

08/10/2021, 3:14 PM

I suspected that I'd need some class of key to get in to that server. And there is a key argument, although I seem to be able to connect to the server without a key.

Philip MacMenamin

08/10/2021, 3:15 PM

I tried to gen a key similar to the mutation listed here https://docs.prefect.io/orchestration/concepts/api_keys.html#using-api-keys but it's not clear to me what a user_id is in this context.

Kevin Kho

08/10/2021, 3:23 PM

You shouldn’t need that for server since there is no auth. Did you configure your

config.toml

to point to <VM-IP>:4200 instead of localhost:4200? This would be done for all machines trying to reach that server (agent or flow registration)

Philip MacMenamin

08/10/2021, 3:49 PM

I did not. I need to do this as well as tell the agent to connect to server IP? I just created a config.toml file such that:

Copy code

$ cat .prefect/config.toml 
api = "http://<server_IP>:4200"

Kevin Kho

08/10/2021, 3:50 PM

Here is the right syntax for that

config.toml

Philip MacMenamin

08/10/2021, 3:59 PM

ok, currently

Copy code

# cat  ~/.prefect/config.toml 
[server]
  [server.ui]
  apollo_url="http://<server_IP>:4200/graphql"

I stared the agent with

Copy code

prefect agent local start --api http://<server_IP>:4200

I deleted the project with the CLI (on the machine running the agent), created the project and registered a flow to that project. The agent is visible to the server, but there are no flows visible.

Kevin Kho

08/10/2021, 4:00 PM

Did you register with the same configuration also?

Philip MacMenamin

08/10/2021, 4:16 PM

yes, I registered the project and flow with that config file in place.

Kevin Kho

08/10/2021, 4:17 PM

Did you restart your server by chance?

Kevin Kho

08/10/2021, 4:17 PM

Were there logs when you registered? Can you show me the output of

prefect diagnostics

Philip MacMenamin

08/10/2021, 4:18 PM

I didn't restart the server, no. I'll get the reg logs for you now.

Philip MacMenamin

08/10/2021, 4:19 PM

Copy code

root@ip-172-20-10-198:/lib/python3.8/nih_3d_resource_workflows# prefect create project "NIH_3D"
NIH_3D created
root@ip-172-20-10-198:/lib/python3.8/nih_3d_resource_workflows# prefect register --project "NIH_3D" -p chimera_x_flow/
Collecting flows...
Processing 'chimera_x_flow/flow.py':
  Building `Local` storage...
  Registering 'ChimeraX_Flow'... Done
  └── ID: 64c7c841-20ec-4d46-95a8-545485271344
  └── Version: 1
======================== 1 registered ========================

Philip MacMenamin

08/10/2021, 4:22 PM

Copy code

root@ip-172-20-10-198:~# prefect diagnostics
{
  "config_overrides": {
    "server": {
      "ui": {
        "apollo_url": true
      }
    }
  },
  "env_vars": [],
  "system_information": {
    "platform": "Linux-5.8.0-1035-aws-x86_64-with-glibc2.29",
    "prefect_backend": "server",
    "prefect_version": "0.15.3",
    "python_version": "3.8.10"
  }
}

Philip MacMenamin

08/10/2021, 4:24 PM

stopped and start prefect server s/w, no change.

Kevin Kho

08/10/2021, 4:30 PM

Oh I was asking because Prefect Server doesn’t keep state by default and you have to use the

--use-volume

to persist the state. This all looks good though. Can I see a screenshot of your UI? Like the dashboard?

Kevin Kho

08/10/2021, 7:15 PM

For posterity, the config.toml should be:

Copy code

[server]
endpoint = "<http://XX.XXX.XXX.XX:4200/graphql>"

👀 1

👍 2

Philip MacMenamin

08/10/2021, 7:22 PM

Great, thanks Kevin!

Open in Slack

Previous Next