Hello everyone :raised_hands: I'm having a problem...
# ask-community
g
Hello everyone 🙌 I'm having a problem with scheduling flows, all of my flows did not schedule automatically overnight and I can't activate them through the interactive API (When I try to
set_schedule_active
I always get
"success": false
). When I try to manually toggle the flow through the GUI the entire server hangs until I manually kill the GraphQL process on the server. Can anyone help or provide some insight? I'm acting as an interpreter for the tech team so I have to do some back and forth with my questions.
🆘 1
👀 1
k
Hey @Gonzalo, that’s weird. It seems to be performance related on the GraphQL pod? This is Prefect Server right? Are all of your pods healthy? Is the db running out of memory?
upvote 1
g
Hey Kevin, yes, it is indeed Prefect Server. We're using Fargate Agent, not Kubernetes, so we're not sure what to answer regarding pods. The tech team says they're not sure what you mean when you refer to a db running out of memory.
I just recieved this error message from them
z
@Gonzalo it looks like they're trying to start Prefect Server but port 5432 is already allocated, so Postgres is unable to start
upvote 1
If Prefect Server isn't running properly, you will not be able to edit schedules
g
Understood, I'll relay the information and I hope that solves it, thanks for the heads up!
👍 1
They're still having trouble. They restarted Prefect entirely, the following error occurred upon running and none of the agents are running right now
k
The agent not spinning up makes sense if server doesn’t spin up. Did all of the containers spin up healthy when you did
prefect server start
? Still seems like something may have not spin up right
g
They're checking the logs right now
All the containers seem to be ok, this is the latest error
k
This is when starting an agent right? Are you able to access the UI/dashboard?
g
Yes, and yes I can access the dashboard
k
Can you try
Copy code
query { hello }
in the interactive API?
g
It returns
Copy code
{
  "data": {
    "hello": "👋"
  }
}
k
And when you query for flows, you see stuff?
Copy code
query {flow {
  id
}}
g
I see several IDs
k
What kind of agent are you starting? Is that a docker agent?
g
It's a Fargate agent
k
Can you show me your config toml? just omit sensitive info
g
One second
Copy code
[server]
  endpoint = "http://<name>:4200"
  request_timeout = 60
  [server.ui]
    apollo_url = "http://<name>:4200/graphql"
where "<name>" is the url
k
Can you try adding
/graphql
after endpoint?
Is this the config.toml of the server or agent? I meant for the agent
g
That's the server's file, they're telling me the agent has no .toml
k
Is the server on the same machine as the agent?
g
The server is on an EC2 instance and the agent is on an ECS machine
(I'm deeply sorry for this cumbersome game of telephone, but I'm also very grateful for your time)
k
So with this walkthrough, you want
Copy code
[server]
  [server.ui]
    apollo_url = "<http://YOUR_MACHINES_PUBLIC_IP:4200/graphql>"
on the server and
Copy code
[server]
endpoint = "YOUR_MACHINES_PUBLIC_IP:4200/graphql"
for the agent. and also
prefect backend server
of course. Really weird if your agent has no config.toml cuz it looks like it’s hitting a nearly correct endpoint there.
And of course! no worries
g
I think we're not really understanding the config.toml as far as the agent goes, we only have one for the server (we tried adding "/graphql" to the end of the endpoint url to no avail).
We have a backend.toml which reads:
Copy code
backend = "server"
that's on the same directory
k
You can make a
config.toml
for the agent also in that same
.prefect
directory
g
I think we found something, is GraphQL all on port 4200 by default? Some of our environment variables (such as GRAPHQL_SERVICE_PORT) point to 4201 instead
Also, making a
config.toml
for the agent under the same directory is confusing since we already have a file with that same name
k
Not inside your working directory. Inside the
.prefect
directory where the
backend.toml
is? You already have a config? GraphQL on port 4201 looks good. But your endpoint will be on 4200
They are separate here
4200 is apollo, 4201 is the graphql
upvote 1
g
Yeah, there's already a
config.toml
there
z
@Gonzalo are you able to edit the existing
.prefect/config.toml
file on the agent ECS machine to include Kevin's addition?
Copy code
[server]
endpoint = "YOUR_MACHINES_PUBLIC_IP:4200/graphql"
If you do have access to an interactive session on the agent machine, can you also confirm the machine can ping the server? This would rule out AWS networking issues that might be blocking the connection
upvote 1