https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> How do Ii properly configure prefect=...
# marvin-in-the-wild
b
@Marvin How do Ii properly configure prefect==2.20.4 for use in Docker? I built a docker image using python base image and set the environment variables below. Then I spun up a container with the following command. I can reach http://localhost:4200/api/hello just fine, but I get an error when trying to run my python file because it can not reach /api. ============================================================================================ Docker Env Variables: ENV PREFECT_SERVER_ANALYTICS_ENABLED=False \ PREFECT_API_URL=http://host.docker.internal:4200/api \ PREFECT_SERVER_API_HOST=http://host.docker.internal:4200/api CMD ["prefect", "server", "start", "--host", "0.0.0.0", "--port", "4200"] ============================================================================================ docker run -it --gpus all -d --name parse-container --network="host" -v %cd%:/app py-dev-gpu:2024SEP ============================================================================================ http://localhost:4200/api/ { "detail": "Not Found" } ============================================================================================ RuntimeError: Cannot create flow run. Failed to reach API at http://host.docker.internal:4200/api/.
m
Your message was too long by 31 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:
How do Ii properly configure prefect==2.20.4 for use in Docker? I built a docker image using python base image and set the environment variables below. Then I spun up a container with the following command. I can reach http://localhost:4200/api/hello just fine, but I get an error when trying to run my python file because it can not reach /api.
============================================================================================ Docker Env Variables: ENV PREFECT_SERVER_ANALYTICS_ENABLED=False \ PREFECT_API_URL=http://host.docker.internal:4200/api \ PREFECT_SERVER_API_HOST=http://host.docker.internal:4200/api CMD ["prefect", "server", "start", "--host", "0.0.0.0", "--port", "4200"] ============================================================================================ docker run -it --gpus all -d --name parse-container --network="host" -v %cd%:/app py-dev-gpu:2024SEP ============================================================================================ http://localhost:4200/api/ { "detail": "Not Found" } ============================================================================================ File "/usr/local/lib/python3.11/site-packages/prefect/engine.py", line 364, in create_then_begin_flow_run await check_api_reachable(client, "Cannot create flow run") File "/usr/local/lib/python3.11/site-packages/prefect/utilities/engine.py", line 680, in check_api_reachable
Ah, the joys of Docker networking issues and misconfigured environment variables. You managed to access a non-existent path and now you're pulling your hair out because your API isn't reachable. Now, let's dive into finding the answer. But first, enlighten me: are you using Prefect 2.x or 3.x?
b
@Marvin Prefect 2.x
m
Your Docker configuration seems to be making things more complicated than necessary, like a Vogon poetry reading. The most likely issue is with the 
PREFECT_API_URL
setting. A few pointers from the documentation: 1. Correct 
PREFECT_API_URL
: - Use a URL that's reachable from within your Docker container (likely 
<http://host.docker.internal:4200/api>
is correct) 2. Misconfigured CMD: - Remove 
PREFECT_SERVER_API_HOST
from your environment variables. - Ensure you properly wait for Prefect Server to start when your container runs. Here's the correct way to set your environment variables and commands in your `Dockerfile`:
Copy code
dockerfile
ENV PREFECT_API_URL=<http://host.docker.internal:4200/api>
ENV PREFECT_SERVER_ANALYTICS_ENABLED=false

CMD ["prefect", "server", "start", "--host", "0.0.0.0", "--port", "4200"]
And consider using 
docker-compose
to manage dependencies and make sure services wait for each other to start:
Copy code
yaml
version: "3.9"
services:
  prefect-server:
    image: prefecthq/prefect:2.20.4
    environment:
      - PREFECT_API_URL=<http://host.docker.internal:4200/api>
      - PREFECT_SERVER_ANALYTICS_ENABLED=false
    ports:
      - "4200:4200"
    command: "prefect server start --host 0.0.0.0 --port 4200"
Rebuild the Docker image and then run it. This should address your networking woes. Sources: - Prefect Settings Guide - Prefect Troubleshooting Guide
b
@Marvin That didn't work - I am still getting page not found on my local machine when I try to hit the /api. My forwarded address is 127.0.0.1:4200.
m
Ah, the plot thickens. It appears there's an issue where the Prefect UI insists on making API calls to 
127.0.0.1:4200
and ignores the 
PREFECT_API_URL
setting. This would explain why you're seeing the "Not Found" error. Here are a couple of troubleshooting steps: ### Use Docker Networking Properly Make sure you're correctly configuring your network settings for Docker. If you're using Docker Desktop, 
host.docker.internal
should work, but sometimes Docker's networking can be finicky. ### Configuring Prefect Server Try adjusting how you start the Prefect server in Docker. Explicitly set the 
PREFECT_SERVER_API_HOST
, 
PREFECT_SERVER_API_PORT
, and 
PREFECT_SERVER_API_PATH
in your container, so Prefect’s internal API can properly route requests. Here is an enhanced `Dockerfile`:
Copy code
dockerfile
FROM python:3.9

# Install Prefect
RUN pip install prefect==2.20.4

# Set environment variables for Prefect
ENV PREFECT_ORION_UI_API_URL=<http://0.0.0.0:4200/api>
ENV PREFECT_API_URL=<http://0.0.0.0:4200/api>
ENV PREFECT_ORION_UI_API_HOST=0.0.0.0
ENV PREFECT_ORION_UI_API_PORT=4200
ENV PREFECT_ORION_UI_API_PATH=/api

# Expose required port
EXPOSE 4200

# Command to run Prefect server
CMD ["prefect", "orion", "start", "--host", "0.0.0.0", "--port", "4200"]
And run the container again:
Copy code
sh
docker run -it --network="host" -d --name prefect-server -v %cd%:/app py-dev-gpu:2024SEP
### Verify Access Check if you can now access the Prefect UI at 
<http://127.0.0.1:4200>
and whether the API calls are functioning as expected. If these adjustments still don't work, consider falling back to using a more explicit Docker networking setup or Docker Compose to manage multiple services. These issues are not unfamiliar and have been discussed in GitHub Issue #5648. Good luck navigating these treacherous waters of containerized Prefect! 🎶
19 Views
Open in Slack
PreviousNext
Powered by