https://prefect.io logo
Title
m

Mitchell Bregman

04/10/2020, 10:28 PM
Hey there! I was able to setup the Prefect Local UI just fine on an EC2... However, when I register the flow, it does not show up in the UI and "Upcoming Runs" gets stuck loading... I was wondering if anyone else is experiencing this? It seems like the flow UUID gets picked up just fine on the backend local server and when I
flow.register
but nothing seems to be showing up
c

Chris White

04/10/2020, 10:50 PM
Hi @Mitchell Bregman could you describe more what you’re seeing? Questions I have: - what happens when you call
flow.register
and what URL is printed? - have you run
prefect backend server
to ensure your are pointed to your local server vs. Cloud?
m

Mitchell Bregman

04/10/2020, 10:53 PM
Hi Chris! Yes, I ran
prefect backend server
before setting up the prefect local server (i.e.
prefect start server
).. When I call
flow.register
, this is the result:
$ python <http://flow.py|flow.py> 
Result Handler check: OK
Flow: <http://localhost:8080/flow/9a703825-7ade-4d92-afc9-d3f8019bf933>
To note: I exposed port
8080
on AWS so that I can access the URL. On the backend, after running the flow registration, I end up seeing nothing being scheduled, i.e.
scheduler_1  | [2020-04-10 22:49:59,153] INFO - prefect-server.Scheduler | Scheduled 0 flow runs.
scheduler_1  | [2020-04-10 22:49:59,254] DEBUG - prefect-server.Scheduler | Sleeping for 300.0 seconds...
c

Chris White

04/10/2020, 10:54 PM
Hmmm interesting - could you run:
from prefect import Client; c = Client()

print(c.api_server)

print(c.graphql("query{flow{name}}")
m

Mitchell Bregman

04/10/2020, 10:56 PM
Yup:
Python 3.7.7 (default, Mar 26 2020, 15:48:22) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from prefect import Client; c = Client()
>>> 
>>> print(c.api_server)
<http://localhost:4200/graphql/alpha>
>>> print(c.graphql("query{flow{name}}"))
{'data': {'flow': [{'name': 'ETL'}, {'name': 'ETL'}]}, 'extensions': {'tracing': {'version': 1, 'startTime': '2020-04-10T22:55:38.693Z', 'endTime': '2020-04-10T22:55:38.700Z', 'duration': 6503370, 'execution': {'resolvers': [{'path': ['flow'], 'parentType': 'Query', 'fieldName': 'flow', 'returnType': '[flow!]!', 'startOffset': 145955, 'duration': 6268650}, {'path': ['flow', 0, 'name'], 'parentType': 'flow', 'fieldName': 'name', 'returnType': 'String!', 'startOffset': 6435474, 'duration': 27895}, {'path': ['flow', 1, 'name'], 'parentType': 'flow', 'fieldName': 'name', 'returnType': 'String!', 'startOffset': 6477149, 'duration': 12499}]}}}}
>>>
c

Chris White

04/10/2020, 10:56 PM
That all seems correct; and your Flow has a schedule attached to it?
m

Mitchell Bregman

04/10/2020, 10:57 PM
For this example, I'm using the ETL example thats in the video above.. no schedule attached to this flow.. but before trying the example ETL flow, I built my own that had a schedule attached - same result
It seems to be working fine, but just not showing up on the UI
c

Chris White

04/10/2020, 10:59 PM
so when you navigate to the Flows tab, it’s just empty?
m

Mitchell Bregman

04/10/2020, 11:00 PM
yup -
c

Chris White

04/10/2020, 11:01 PM
you mentioned running on an EC2 instance and port-forwarding; this might be a dumb question but are you sure you don’t accidentally have two server instances spun up? One on your local machine and one on the EC2?
m

Mitchell Bregman

04/10/2020, 11:03 PM
i never set up on server locally.. i set up the server on EC2 in one
tmux
screen and am running the flow in another
c

Chris White

04/10/2020, 11:04 PM
I assume refreshing that screen doesn’t help? if you navigate to the interactive API and run the same query:
query{
   flow{ name }
}
does anything show up?
m

Mitchell Bregman

04/10/2020, 11:05 PM
the result from that query:
{
  "graphQLErrors": [],
  "networkError": {},
  "message": "Network error: Failed to fetch"
}
c

Chris White

04/10/2020, 11:05 PM
ahhhh ok ok now we’re getting somewhere
so for some reason the UI can’t talk to Apollo, but your python session can 🧐
m

Mitchell Bregman

04/10/2020, 11:06 PM
hmmm. sorry if this is dumb but i assume
name
gets substituted with
ETL
? regardless, same result in that query
c

Chris White

04/10/2020, 11:07 PM
yea it should return the same thing as the python client returned
m

Mitchell Bregman

04/10/2020, 11:07 PM
right.. i wonder where the disconnect is
c

Chris White

04/10/2020, 11:08 PM
and you ran
prefect server start
with no configuration?
m

Mitchell Bregman

04/10/2020, 11:08 PM
i tried setting up the server from source as well, same issue
yes
c

Chris White

04/10/2020, 11:08 PM
OOO wow I’m sorry I should’ve caught this earlier
when you port forward the UI to your local machine, the actual code running in your browser is attempting to connect to
localhost:4200
which isn’t connected to your EC2
you can also port forward 4200 and then you should be good
m

Mitchell Bregman

04/10/2020, 11:09 PM
alrighty, will test that out in 1 second
btw - thank you sooooo so mcuh
c

Chris White

04/10/2020, 11:10 PM
of course! anytime
m

Mitchell Bregman

04/10/2020, 11:13 PM
ok, so i set up the port forwarding wiht same rules as
8080
-- do i need to run anything before restarted the server?
c

Chris White

04/10/2020, 11:13 PM
i don’t think so
m

Mitchell Bregman

04/10/2020, 11:14 PM
alrighty - still the same 😕 same with the interactive API call
do i need to expose any other ports...
4201
?
c

Chris White

04/10/2020, 11:15 PM
should just be 4200 and 8080 🧐
m

Mitchell Bregman

04/10/2020, 11:16 PM
hmmm... i can set up port forwarding for anywhere so that you can check it out as well, but not sure if thats helpful lol
@Chris White - let me know if you have a chance to revisit this; cant seem to resolve - thank you!
c

Chris White

04/14/2020, 5:19 PM
Hi @Mitchell Bregman — I believe the issue is that the UI is always going to make queries to
localhost:4200
on the machine it’s running on, so I believe that is the root cause. @nicholas might have some more insight here though and can confirm / deny my theory
🙌 1
n

nicholas

04/14/2020, 5:30 PM
@Chris White @Mitchell Bregman that's correct - I think the simplest way to get around that would be to modify your
/etc/hosts
file to direct
localhost:4200
traffic to your EC2
:upvote: 2
To further clarify, on Mac/Linux you'll want to append to your
/etc/hosts
file an entry that looks like this:
your.ec2.endpoint localhost:4200
You can do that nicely with this command:
echo 'your.ec2.endpoint localhost:4200' >> /etc/hosts
If you're on Windows, I believe the location of the hosts file is
C:\windows\system32\drivers\etc\hosts
m

Manuel Aristarán

04/14/2020, 8:13 PM
Hrm…I’m dealing with something similar (just deployed prefect server in an ECS cluster). @nicholas
/etc/hosts
it’s for name resolution only AFAIK — I’m using
socat
for redirecting localhost traffic to my ECS service
specifying a port number in
/etc/hosts
won’t work
n

nicholas

04/14/2020, 9:56 PM
You're right @Manuel Aristarán - it seems like the
/etc/hosts
hack isn't a great solution - working on a PR that will address this!
m

Mary Clair Thompson

04/20/2020, 6:31 PM
Hi folks! I'm new to this community and working on evaluating Prefect as our new tool for job automation. I set up the server on a vm and encountered exactly the same issue as @Mitchell Bregman when I tried to access the web UI from my machine via an ssh tunnel. I can work around it with the additional tunnel to port 4200 on the server, but I'd love to see a better solution. Can you all give me any info about the status of the PR mentioned by @nicholas above?
n

nicholas

04/20/2020, 6:38 PM
Hi @Mary Clair Thompson and welcome! The PR has been merged to master, I believe it’s slated for release this week (@Chris White can confirm)
👍 1
c

Chris White

04/20/2020, 6:38 PM
Yup, it should be released tomorrow with 0.10.3, along with a blog post (by @nicholas) on how to set things up
🚀 2
m

Mary Clair Thompson

04/20/2020, 6:39 PM
@Chris White wonderful, thank you!
👍 1
m

Mitchell Bregman

04/21/2020, 7:39 PM
@nicholas Thank you so much for all your help and hard work resolving this. This article is fantastic: https://medium.com/the-prefect-blog/prefect-server-101-deploying-to-google-cloud-platform-47354b16afe2 .. PS - Thank you to the whole Prefect team!! You guys are amazing 🙌
😄 2
🚀 2
a

Adrien Boutreau

05/08/2020, 3:14 PM
I have same problem here - did you find a way to solve it ?
{
"graphQLErrors": [],
"networkError": {},
"message": "Network error: NetworkError when attempting to fetch resource."
}
from api
and from python job
http://localhost:4200/graphql/alpha {'data': {'flow': [{'name': 'Zoom_test'}, {'name': 'Zoom_test'}, {'name': 'Zoom_test'}, {'name': 'Zoom_test'}, {'name': 'Zoom_test'}]}}
n

nicholas

05/08/2020, 3:21 PM
@Adrien Boutreau would you mind asking this in a new thread in #prefect-community so we don't spam everyone in this thread?
a

Adrien Boutreau

05/08/2020, 3:22 PM
👍