https://prefect.io logo
Title
i

Ivan Ksaver Šušnjara

05/24/2023, 12:07 PM
Hi All, I have a question regarding accessing Prefect server UI, with Prefect hosted on AWS EC2 Ubuntu 22.04 in private subnet. I have trouble accessing it directly via private IP of instance, while connected to corporate network/VPN, which allows communucating with AWS VPC using private IPs. If anyone has similar experience, I'd appreciate help. Thank you! 🙂 Let me explain the issue in details: Assumptions: • Prefect is hosted on EC2 in private subnet of VPC. It is hosted directly within conda environment, and uses Postgres DB backend, running on Docker container on same instance. This all works fine • I was able to access Prefect UI using a proxy EC2 instance with public IP, which is allowed to send HTTP requests to private IP of the instance Prefect is hosted on. I used following configuration, and everything works great: ◦ prefect config set PREFECT_API_URL='<http😕/1:4200/api|http😕/&lt;private-ip-of-instance&gt;:4200/api>' ◦ prefect config set PREFECT_SERVER_API_HOST='0.0.0.0' ◦ prefect config set PREFECT_API_DATABASE_CONNECTION_URL="<postgresql+asyncpg😕/p:yourTopSecretPassword@localhost:5432/prefect|postgresql+asyncpg://<user>:<pw>@localhost:5432/prefect>" • However, when trying to access '<http😕/1:4200/api|http😕/&lt;private-ip-of-instance&gt;:4200/api>' directly (without proxy) from corporate network/VPN with the same settings as above, Prefect UI complains it can't communicate with API • If I delete PREFECT_API_URL and PREFECT_SERVER_API_HOST from config, and run 'prefect server start --host 0.0.0.0', I'm able to access the UI only if I expose a port in VS Code, and click on attached 'Open in Browser' option. Again everything works fine • However, I want to be able to access UI directly on '<http😕/1:4200/api|http😕/&lt;private-ip-of-instance&gt;:4200/api>' from corporate network/VPN. When I try to do that, I only get a blank page titled 'Prefect Server', like in attached image • Using cURL on <http😕/1:4200/api|http😕/&lt;private-ip-of-instance&gt;:4200> returns HTML response from Prefect • Nothing in network settings, security groups etc changed meanwhile If someone has experience with problems like this, I would really appreciate help. Thank you! 🙂
c

Christopher Boyd

05/24/2023, 12:56 PM
Are you trying to access the API or are you trying to access the web page ? You’d get the same behavior on prefect cloud - they are separate end points
The /api is to interact with the API via rest
image.png
i

Ivan Ksaver Šušnjara

05/24/2023, 1:00 PM
I'm trying to access the web page, This /api suffix probably caused confusion, but I get same results if using /flows or /flow-runs paths, or no path at all
c

Christopher Boyd

05/24/2023, 1:02 PM
what does the browser console show?
This suggests to me something with your VPN and firewall:
I was able to access Prefect UI using a proxy EC2 instance with public IP, which is allowed to send HTTP requests to private IP of the instance Prefect is hosted on. I used following configuration, and everything works great:
prefect config set PREFECT_API_URL='http://<private-ip-of-instance>:4200/api'
prefect config set PREFECT_SERVER_API_HOST='0.0.0.0'
prefect config set PREFECT_API_DATABASE_CONNECTION_URL="postgresql+asyncpg://<user>:<pw>@localhost:5432/prefect"
However, when trying to access 'http://<private-ip-of-instance>:4200/api' directly (without proxy) from corporate network/VPN with the same settings as above, Prefect UI complains it can't communicate with API
the fact that you can do it via an external IP to EC2 that bypasses your VPN works, but using your vpn doesn’t work tells me it’s not an issue with the server
I would check in the console to see if pages are being retrieved, and check the server logs to see if requests are being received and returned
i

Ivan Ksaver Šušnjara

05/24/2023, 1:08 PM
sure, I can do that, thanks for the advice! 🙂 strange thing though is that some other webpages hosted on same EC2 instance can be accessed from corporate network, using private IP and respective port my idea was that something was incorrectly set up in Prefect config, and it was not clear to me what that is