https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I have setup a .env file with the bel...
# marvin-ai
a
@Marvin I have setup a .env file with the below for basic auth PREFECT_SERVER_API_AUTH_STRING="admin:password" PREFECT_API_AUTH_STRING="admin:password" However my worker doesn't run with this method, but if I hardcode the admin:password in the docker compose file it does work. Any ideas?
m
✅ thought for 276.9 seconds
A couple likely gotchas here: - In Docker Compose, the top-level .env file is only used for variable interpolation in the compose file; it does not automatically inject those variables into your containers unless you reference them or use 
env_file:
. Also, quotes in 
.env
are not stripped — they become part of the value. - For Prefect 3.x, your server needs 
PREFECT_SERVER_API_AUTH_STRING
and your worker needs 
PREFECT_API_AUTH_STRING
. The worker also needs a valid 
PREFECT_API_URL
. What to try: 1) Remove quotes in your .env Your example shows quotes. Use this instead:
Copy code
PREFECT_SERVER_API_AUTH_STRING=admin:password
PREFECT_API_AUTH_STRING=admin:password
2) Make sure the env actually reaches your containers Use either env_file or environment interpolation. Option A: env_file (passes everything in .env to the container)
Copy code
services:
  prefect-server:
    image: prefecthq/prefect:3-latest
    env_file:
      - .env
    environment:
      - PREFECT_SERVER_API_AUTH_STRING=${PREFECT_SERVER_API_AUTH_STRING}
      - PREFECT_UI_API_URL=<http://prefect-server:4200/api>
    # ...

  worker:
    image: prefecthq/prefect:3-latest
    env_file:
      - .env
    environment:
      - PREFECT_API_URL=<http://prefect-server:4200/api>
      - PREFECT_API_AUTH_STRING=${PREFECT_API_AUTH_STRING}
    # ...
Option B: explicit interpolation without env_file
Copy code
services:
  worker:
    environment:
      - PREFECT_API_URL=<http://prefect-server:4200/api>
      - PREFECT_API_AUTH_STRING=${PREFECT_API_AUTH_STRING}
Important: - Keep 
.env
in the same directory as your compose file (or pass it explicitly). Compose only auto-loads 
.env
from the project directory. - Don’t include quotes in 
.env
. They’ll be included as literal characters and break auth. - Confirm the server also has 
PREFECT_SERVER_API_AUTH_STRING
set to the same value. 3) Verify inside the container After 
docker compose up
, check the worker:
Copy code
docker compose exec worker env | grep PREFECT_API_
docker compose exec worker prefect config view | sed -n "/api:/,/^$/p"
You should see 
api.auth_string
populated, and 
api.url
pointing to your server. If it still fails: - Please share the relevant bits of your docker-compose.yml for the server and worker. - Confirm where your 
.env
is located relative to the compose file. - Paste the first error lines from the worker logs (e.g., 401s). Reference: Compose .env file semantics (quotes, location, and behavior) — Compose .env file docs
a
@Marvin
Copy code
version: "3.9"
services:

  ### Prefect Database
  database:
    image: postgres:15.2-alpine
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=prefect
    expose:
      - 5432
    volumes:
      - db:/var/lib/postgresql/data
    profiles: ["server"]

  ### Prefect Server API and UI
  server:
    image: prefecthq/prefect:3.4-python3.11
    restart: always
    volumes:
      - prefect:/root/.prefect
    entrypoint: sh -c "/opt/prefect/entrypoint.sh prefect server start"
    environment:
      - PREFECT_UI_URL=<http://example:4200/api>
      - PREFECT_API_URL=<http://example:4200/api>
      - PREFECT_SERVER_API_HOST=0.0.0.0
      - PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://postgres:postgres@database:5432/prefect>
      - PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
    ports:
      - 4200:4200
    depends_on:
      - database
    profiles: ["server"]

  ## Prefect worker
  worker:
    build:
      context: .
      dockerfile: dockerfile
    restart: always
    volumes:
      - "./projects:/root/projects"
    entrypoint: sh -c "/opt/prefect/entrypoint.sh prefect worker start --pool default-work-pool"
    environment:
      - PREFECT_API_URL=<http://server:4200/api>
      - PREFECT_API_AUTH_STRING=${AUTH_STRING}
    profiles: ["worker"]

volumes:
  prefect:
  db:
networks:
  default:
    name: prefect-network
m
✅ thought for 43.5 seconds
Thanks for sharing the compose file — the issue is the env var name mismatch. In your compose, you reference 
${AUTH_STRING}
for both the server and worker, but in your 
.env
you set 
PREFECT_SERVER_API_AUTH_STRING
and 
PREFECT_API_AUTH_STRING
. Since 
AUTH_STRING
is not defined, it expands to empty and the worker can’t auth. When you hardcode it, it works — consistent with this. Also, remove quotes in your 
.env
(they become part of the value) and make sure you’re not literally including angle brackets around URLs. What to change: - Either define a single 
AUTH_STRING
in `.env`:
Copy code
AUTH_STRING=admin:password
- Or reference the correct variables directly in compose: - Server: 
PREFECT_SERVER_API_AUTH_STRING=${PREFECT_SERVER_API_AUTH_STRING}
- Worker: 
PREFECT_API_AUTH_STRING=${PREFECT_API_AUTH_STRING}
And fix the UI var name and URLs (no angle brackets). Here’s a corrected compose example using a single 
AUTH_STRING
in `.env`:
Copy code
version: "3.9"
services:

  database:
    image: postgres:15.2-alpine
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=prefect
    expose:
      - 5432
    volumes:
      - db:/var/lib/postgresql/data
    profiles: ["server"]

  server:
    image: prefecthq/prefect:3.4-python3.11
    restart: always
    volumes:
      - prefect:/root/.prefect
    entrypoint: sh -c "/opt/prefect/entrypoint.sh prefect server start"
    environment:
      - PREFECT_UI_API_URL=<http://server:4200/api>
      - PREFECT_API_URL=<http://server:4200/api>
      - PREFECT_SERVER_API_HOST=0.0.0.0
      - PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://postgres:postgres@database:5432/prefect>
      - PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
    ports:
      - 4200:4200
    depends_on:
      - database
    profiles: ["server"]

  worker:
    build:
      context: .
      dockerfile: dockerfile
    restart: always
    volumes:
      - "./projects:/root/projects"
    entrypoint: sh -c "/opt/prefect/entrypoint.sh prefect worker start --pool default-work-pool"
    environment:
      - PREFECT_API_URL=<http://server:4200/api>
      - PREFECT_API_AUTH_STRING=${AUTH_STRING}
    profiles: ["worker"]

volumes:
  prefect:
  db:

networks:
  default:
    name: prefect-network
And your `.env`:
Copy code
AUTH_STRING=admin:password
If you prefer separate vars, use: - In server: 
PREFECT_SERVER_API_AUTH_STRING=${PREFECT_SERVER_API_AUTH_STRING}
- In worker: 
PREFECT_API_AUTH_STRING=${PREFECT_API_AUTH_STRING}
and set both in 
.env
(without quotes). Verify inside the containers:
Copy code
docker compose exec worker env | grep PREFECT_API_
docker compose exec worker prefect config view | sed -n "/api:/,/^$/p"
You should see 
api.url
set to 
<http://server:4200/api>
and 
api.auth_string
populated. Compose .env behavior reference: Compose .env file docs If it still fails, can you share the first error lines from the worker logs (e.g., 401 Unauthorized) and confirm your 
.env
lives next to the compose file?
Sorry, I encountered an error while processing your request. Please try again.
a
@Marvin My .env file is just
Copy code
AUTH_STRING=admin:password
and lives next to the compose file
m
✅ thought for 80.0 seconds
✅ thought for 81.4 seconds
✅ thought for 66.1 seconds
Got it — that 
AUTH_STRING
value looks fine. The most common remaining pitfalls are: - The compose file still has angle brackets around URLs or DSNs. Make sure your actual compose file does not include 
<
and 
>
around values. - Containers aren’t actually receiving the resolved value from 
.env
. - Server and worker end up with different values, or server is missing 
PREFECT_SERVER_API_AUTH_STRING
. Quick checklist 1) Ensure compose references match your .env - Server:
Copy code
- PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
- Worker:
Copy code
- PREFECT_API_AUTH_STRING=${AUTH_STRING}
2) Remove angle brackets from URLs Example:
Copy code
- PREFECT_API_URL=<http://server:4200/api>
# (and if you set it on the server container too)
- PREFECT_API_URL=<http://server:4200/api>
3) Recreate containers to pick up .env updates
Copy code
docker compose up -d --force-recreate
4) Verify the server and worker both see the same values - Worker:
Copy code
docker compose exec worker env | grep PREFECT_API_
docker compose exec worker prefect config view | sed -n "/api:/,/^$/p"
- Server:
Copy code
docker compose exec server env | grep PREFECT_SERVER_API_
docker compose exec server prefect config view | sed -n "/server:/,/^$/p"
You should see: - worker: 
api.url=<http://server:4200/api>
and 
api.auth_string
populated - server: 
server.api.auth_string
populated 5) If it still fails, grab first errors from worker logs
Copy code
docker compose logs -n 100 worker
If you see 401s, the server likely doesn’t have the same auth_string set. Optional sanity test from worker
Copy code
docker compose exec worker sh -lc 'python - <<PY
import httpx, base64
u="<http://server:4200/api/health>"
auth="admin:eyjllSdMQDP55kHawLPW"
h={"Authorization":"Basic " + base64.b64encode(auth.encode()).decode()}
print(httpx.get(u, headers=h, timeout=5).status_code)
PY'
Expect 200. If not, the server auth isn’t matching. If you want, paste: - The exact environment sections for both services after you’ve removed angle brackets - The first 20 lines of 
docker compose logs worker
after start Happy to spot-check!
Got it — 
AUTH_STRING=admin:password
in your .env is fine. Quick checklist to get this working end-to-end: 1) Verify Compose is actually reading .env and expanding ${AUTH_STRING} - From the same directory as your compose file and .env, run:
Copy code
docker compose config | grep AUTH_STRING -n
You should see the expanded values (not literally ${AUTH_STRING}). 2) Ensure no angle brackets around URLs In compose, use:
Copy code
PREFECT_API_URL=<http://server:4200/api>
PREFECT_UI_API_URL=<http://server:4200/api>
Angle brackets will break it. 3) Confirm the right vars are on the right services - Server needs: 
PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
- Worker needs: 
PREFECT_API_AUTH_STRING=${AUTH_STRING}
4) Inspect inside the running containers
Copy code
docker compose exec server env | grep PREFECT_SERVER_API_AUTH_STRING
docker compose exec worker env | grep PREFECT_API_
docker compose exec worker prefect config view | sed -n "/api:/,/^$/p"
You should see api.url=http://server:4200/api and api.auth_string set. 5) If everything above looks correct but the worker still fails to start/auth: There are reports of auth_string issues in some 3.4.x builds. Try upgrading the image to the latest 3.x: - Change both images to 
prefecthq/prefect:3-latest
(or a newer 3.5+ tag), then:
Copy code
docker compose pull
docker compose up -d
Related reports: - Docker server does not respect PREFECT_API_AUTH_STRING or PREFECT_SERVER_API_AUTH_STRING (3.4.7) - auth_string based authentication isn’t working (3.4.4) Docs for reference: - Run Prefect Server with Docker - Advanced security settings (Basic Auth) If it still doesn’t work, could you paste: - The first error lines from the worker logs (401/403, etc.) - Output of the 
prefect config view
snippet from inside the worker - Confirmation that 
docker compose config
shows AUTH_STRING expanded?
Got it — thanks. Since you’re using 
${AUTH_STRING}
in compose and 
.env
contains 
AUTH_STRING=admin:password
, the most common reasons it still doesn’t work are: 1) Compose isn’t loading your .env - Compose only auto-loads 
.env
from the “project directory” (the directory of the first compose file you pass). - If you run from another directory or use 
-f path/to/compose.yml
, Compose won’t see the 
.env
next to your file unless that directory is the project directory. - Fix by either: - cd into the directory with your compose file and run 
docker compose up
, or - explicitly pass it: 
docker compose --env-file .env up
2) A shell env var is overriding .env - Shell env vars take precedence over 
.env
. - Check: 
echo $AUTH_STRING
(or 
env | grep AUTH_STRING
). If it’s set differently or empty, it will override the 
.env
value. 3) Angle brackets and var names - Make sure your compose file does not literally contain 
<http://...>
— use plain URLs: - 
PREFECT_API_URL=<http://server:4200/api>
- 
PREFECT_UI_API_URL=<http://server:4200/api>
(note the correct var name; not 
PREFECT_UI_URL
) Quick checks to pinpoint it: - Render the final config and verify interpolation happened:
Copy code
docker compose config | sed -n '/server:/,/profiles:/p'
docker compose config | sed -n '/worker:/,/profiles:/p'
You should see:
Copy code
PREFECT_SERVER_API_AUTH_STRING: admin:password
PREFECT_API_AUTH_STRING: admin:password
- Inside the running worker:
Copy code
docker compose exec worker env | grep PREFECT_API_
docker compose exec worker prefect config view | sed -n "/api:/,/^$/p"
You should see 
api.url: <http://server:4200/api>
and 
api.auth_string: admin:password
. If you prefer, you can also switch to referencing the explicit Prefect vars (same result): - Server:
Copy code
- PREFECT_SERVER_API_AUTH_STRING=${PREFECT_SERVER_API_AUTH_STRING}
- Worker:
Copy code
- PREFECT_API_AUTH_STRING=${PREFECT_API_AUTH_STRING}
and put both in 
.env
(no quotes). If it still fails, please share: - The output of 
docker compose config
for the server/worker sections - The first few lines of the worker logs (e.g., 401 errors) - How you’re invoking 
docker compose
(cwd and flags) Compose .env behavior reference: Compose .env file docs
a
@Marvin this is the output for dock compose config for server:
Copy code
server:
    command:
      - prefect
      - server
      - start
    depends_on:
      database:
        condition: service_healthy
        required: true
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://postgres:postgres@database:5432/prefect>
      PREFECT_API_URL: <http://exampleserver:4200/api>
      PREFECT_SERVER_API_AUTH_STRING: admin:pass
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_UI_URL: <http://exampleserver:4200/api>
    image: prefecthq/prefect:3.4-python3.11
    networks:
      default: null
    ports:
      - mode: ingress
        target: 4200
        published: "4200"
        protocol: tcp
    restart: always
    volumes:
      - type: volume
        source: prefect
        target: /root/.prefect
        volume: {}
  worker:
    build:
      context: /home/compose/prefect
      dockerfile: dockerfile
    command:
      - prefect
      - worker
      - start
      - --pool
      - default-work-pool
    depends_on:
      server:
        condition: service_started
        required: true
    environment:
      PREFECT_API_AUTH_STRING: admin:pass
      PREFECT_API_URL: <http://server:4200/api>
    networks:
      default: null
    restart: always
    volumes:
      - type: bind
        source: /home/compose/prefect/projects
        target: /root/projects
        bind:
          create_host_path: true
networks:
  default:
    name: prefect-network
volumes:
  db:
    name: prefect_db
  prefect:
    name: prefect_prefect
and this is the worker output:
Copy code
worker:
    build:
      context: /home/compose/prefect
      dockerfile: dockerfile
    command:
      - prefect
      - worker
      - start
      - --pool
      - default-work-pool
    depends_on:
      server:
        condition: service_started
        required: true
    environment:
      PREFECT_API_AUTH_STRING: admin:pass
      PREFECT_API_URL: <http://server:4200/api>
    networks:
      default: null
    restart: always
    volumes:
      - type: bind
        source: /home/compose/prefect/projects
        target: /root/projects
        bind:
          create_host_path: true
networks:
  default:
    name: prefect-network
volumes:
  db:
    name: prefect_db
  prefect:
    name: prefect_prefect
m
✅ thought for 45.2 seconds
Thanks — that config output helps. Two things jump out: 1) Your URLs/DSN still have angle brackets. In your rendered config I see: - 
PREFECT_API_URL: <http://server:4200/api>
- 
PREFECT_API_URL: <http://exampleserver:4200/api>
- 
PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://...>
Those literal 
<
and 
>
will break the client/server. Remove them so the final rendered config shows plain URLs. 2) Wrong UI env var name. Use 
PREFECT_UI_API_URL
(not 
PREFECT_UI_URL
) to point the UI at the API. A corrected snippet:
Copy code
server:
  image: prefecthq/prefect:3-latest
  environment:
    - PREFECT_SERVER_API_HOST=0.0.0.0
    - PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
    - PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://postgres:postgres@database:5432/prefect>
    - PREFECT_API_URL=<http://exampleserver:4200/api>          # optional
    - PREFECT_UI_API_URL=<http://exampleserver:4200/api>       # correct var name
  ports:
    - "4200:4200"

worker:
  environment:
    - PREFECT_API_URL=<http://server:4200/api>
    - PREFECT_API_AUTH_STRING=${AUTH_STRING}
Then: - Recreate: 
docker compose up -d --force-recreate
- Verify the interpolation and no angle brackets: - 
docker compose config | sed -n '/server:/,/profiles:/p'
- 
docker compose config | sed -n '/worker:/,/profiles:/p'
- Verify inside containers: - 
docker compose exec server env | grep PREFECT_SERVER_API_AUTH_STRING
- 
docker compose exec worker prefect config view | sed -n "/api:/,/^$/p"
Optional: there have been auth_string quirks in 3.4.x; if you still hit odd behavior after fixing URLs/var name, try bumping to 
prefecthq/prefect:3-latest
and `docker compose pull`: - Related reports: Docker server does not respect PREFECT_API_AUTH_STRING and auth_string based authentication isn’t working Docs: - Run Prefect Server with Docker - Basic auth / security settings If it still fails, please share the first lines from 
docker compose logs worker
and the output of 
prefect config view
inside the worker.
b
@Ashley Kent can you try adding quotes to the values for the AUTH_STRINGs in the docker compose file and the 
.env
file? YAML can be sensitive to colons in values...
Copy code
PREFECT_API_AUTH_STRING: "admin:pass"
a
Hi @Brendan Dalpe the server works fine and I can login OK, but the worker just goes to 
Restarting (1) 48 seconds ago
when I include PREFECT_API_AUTH_STRING. Here is my full docker compose file which is in the same folder as the .env file, AUTH_STRING is just password:
Copy code
services:
  database:
    image: postgres:15.2-alpine
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=prefect
    expose:
      - 5432
    volumes: 
      - db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect"]
      interval: 5s
      timeout: 5s
      retries: 5
  
  server:
    image: prefecthq/prefect:3.4-python3.11
    restart: always
    depends_on:
      database:
        condition: service_healthy
    volumes:
      - prefect:/root/.prefect
    command: prefect server start
    environment:
      - PREFECT_UI_URL=<http://exampleserver:4200/api>
      - PREFECT_API_URL=<http://exampleserver:4200/api>
      - PREFECT_SERVER_API_HOST=0.0.0.0
      - PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://postgres:postgres@database:5432/prefect>
      - PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
    ports:
      - 4200:4200

  worker:
    build: 
      context: .
      dockerfile: dockerfile
    restart: always
    depends_on:
      server:
        condition: service_started
    volumes:
      - "./projects:/root/projects"
    command: prefect worker start --pool default-work-pool
    environment:
      - PREFECT_API_URL=<http://server:4200/api>
      - PREFECT_API_AUTH_STRING=${AUTH_STRING}
    
volumes:
  prefect:
  db:
networks:
  default:
    name: prefect-network
An issue with my docker compose maybe?
b
@Ashley Kent confirming that 
AUTH_STRING=password:
with the colon on the end?
a
@Brendan Dalpe it was just password, if I try AUTH_STRING="admin:password" I get the same issue and the worker fails to start
✅ 1
b
Ok, thanks. Looking at this now.
Can you share your 
Dockerfile
you're using for the worker?
a
Copy code
FROM prefecthq/prefect:3.4-python3.11

RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y nano
RUN apt-get install -y odbcinst

RUN curl <https://public.dhe.ibm.com/software/ibmi/products/odbc/debs/dists/1.1.0/ibmi-acs-1.1.0.list> | tee /etc/apt/sources.list.d/ibmi-acs-1.1.0.list
RUN apt-get update
RUN apt-get install -y ibm-iaccess

WORKDIR /root/projects
thank you 1
b
@Ashley Kent I'm not able to replicate this issue... I have in `.env`:
Copy code
AUTH_STRING=admin:password
I am using your 
docker-compose.yml
file you provided.
Can you give me the exact error you're seeing on your worker please?
a
@Brendan Dalpe
Copy code
database-1  | 2025-08-27 15:24:32.227 UTC [64] FATAL:  role "prefect" does not exist
worker-1    | Traceback (most recent call last):
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 42, in wrapper
worker-1    |     return fn(*args, **kwargs)
worker-1    |            ^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/_types.py", line 153, in sync_fn
worker-1    | An exception occurred.
worker-1    |     return asyncio.run(async_fn(*args, **kwargs))
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
worker-1    |     return runner.run(main)
worker-1    |            ^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
worker-1    |     return self._loop.run_until_complete(task)
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
worker-1    |     return future.result()
worker-1    |            ^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/worker.py", line 114, in start
worker-1    |     is_paused = await _check_work_pool_paused(work_pool_name)
worker-1    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/worker.py", line 179, in _check_work_pool_paused
worker-1    |     work_pool = await client.read_work_pool(work_pool_name=work_pool_name)
worker-1    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/client/orchestration.py", line 2679, in read_work_pool
worker-1    |     response = await self._client.get(f"/work_pools/{work_pool_name}")
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1814, in get
worker-1    |     return await self.request(
worker-1    |            ^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1585, in request
worker-1    |     return await self.send(request, auth=auth, follow_redirects=follow_redirects)
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/client/base.py", line 361, in send
worker-1    |     response.raise_for_status()
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/client/base.py", line 174, in raise_for_status
worker-1    |     raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
worker-1    | prefect.exceptions.PrefectHTTPStatusError: Client error '401 Unauthorized' for url '<http://server:4200/api/work_pools/default-work-pool>'
worker-1    | Response: {'exception_message': 'Unauthorized'}
worker-1    | For more information check: <https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401>
worker-1 exited with code 1
If I remove the 
PREFECT_SERVER_API_AUTH_STRING=${AUTH_STRING}
from server and 
PREFECT_API_AUTH_STRING=${AUTH_STRING}
from worker it seems to start OK (some errors still, not sure what they are...)
Copy code
database-1  | 2025-08-27 15:27:54.077 UTC [41] FATAL:  role "prefect" does not exist
worker-1 exited with code 1
worker-1    | Traceback (most recent call last):
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions
worker-1    |     yield
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 377, in handle_async_request
worker-1    |     resp = await self._pool.handle_async_request(req)
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
worker-1    |     raise exc from None
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
worker-1    |     response = await connection.handle_async_request(
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
worker-1    |     raise exc
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
worker-1    |     stream = await self._connect(request)
worker-1    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 122, in _connect
worker-1    |     stream = await self._network_backend.connect_tcp(**kwargs)
worker-1    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
worker-1    |     return await self._backend.connect_tcp(
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 115, in connect_tcp
worker-1    |     with map_exceptions(exc_map):
worker-1    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
worker-1    |     self.gen.throw(typ, value, traceback)
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
worker-1    |     raise to_exc(exc) from exc
worker-1    | httpcore.ConnectError: All connection attempts failed
worker-1    | 
worker-1    | The above exception was the direct cause of the following exception:
worker-1    | 
worker-1    | Traceback (most recent call last):
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 42, in wrapper
worker-1    |     return fn(*args, **kwargs)
worker-1    |            ^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/_types.py", line 153, in sync_fn
worker-1    |     return asyncio.run(async_fn(*args, **kwargs))
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
worker-1    |     return runner.run(main)
worker-1    |            ^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
worker-1    |     return self._loop.run_until_complete(task)
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
worker-1    |     return future.result()
worker-1    |            ^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/worker.py", line 114, in start
worker-1    |     is_paused = await _check_work_pool_paused(work_pool_name)
worker-1    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/worker.py", line 179, in _check_work_pool_paused
worker-1    |     work_pool = await client.read_work_pool(work_pool_name=work_pool_name)
worker-1    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/client/orchestration.py", line 2679, in read_work_pool
worker-1    |     response = await self._client.get(f"/work_pools/{work_pool_name}")
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1814, in get
worker-1    |     return await self.request(
worker-1    |            ^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1585, in request
worker-1    |     return await self.send(request, auth=auth, follow_redirects=follow_redirects)
worker-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/client/base.py", line 330, in send
worker-1    |     response = await self._send_with_retry(
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/prefect/client/base.py", line 254, in _send_with_retry
worker-1    |     response = await send(request, *send_args, **send_kwargs)
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1674, in send
worker-1    |     response = await self._send_handling_auth(
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
worker-1    |     response = await self._send_handling_redirects(
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
worker-1    |     response = await self._send_single_request(request)
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1776, in _send_single_request
worker-1    |     response = await transport.handle_async_request(request)
worker-1    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 376, in handle_async_request
worker-1    |     with map_httpcore_exceptions():
worker-1    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
worker-1    |     self.gen.throw(typ, value, traceback)
worker-1    |   File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
worker-1    |     raise mapped_exc(message) from exc
worker-1    | httpx.ConnectError: All connection attempts failed
worker-1    | An exception occurred.
worker-1    | Discovered type 'process' for work pool 'default-work-pool'.
worker-1    | Discovered type 'process' for work pool 'default-work-pool'.
worker-1    | Worker 'ProcessWorker 2b9e9b66-6b0d-42e4-8bb6-098ab8df76d8' started!
worker-1    | Worker 'ProcessWorker 2b9e9b66-6b0d-42e4-8bb6-098ab8df76d8' started!
database-1  | 2025-08-27 15:27:59.189 UTC [57] FATAL:  role "prefect" does not exist
For context, in my docker compose under server I am updating "http://exampleserver:4200/api" to my actual server, and in worker I am leaving it as "http://server:4200/api" which seems to work, is this correct? If I update this second part to my actual server name it errors...
b
@Ashley Kent I'm not fully following what you're saying here. You have two Prefect server instances?
a
@Brendan Dalpe no, I was just checking that my docker compose file was correct in hardcoding the value http://server:4200/api in the PREFECT_API_URL under worker. My actual server is defined earlier on under server e.g. http://exampleserver:4200/api
I also fixed the FATAL: role "prefect" does not exist issue as I'd defined the wrong user under the database healthcheck
b
I'm not sure what you mean by "My actual server is defined"?
Is this the endpoint you're using to access your Prefect server? Is this another Prefect server deployed somewhere else?
a
No all one server, I might have confused things sorry
b
I'm still confused where 
<http://exampleserver:4200/api>
fits into the picture here
a
exampleserver is my servername
b
Ok, so you're running the docker compose on a remote machine and then accessing it through your Browser?
a
Correct
✅ 1
b
Yes, in this case you'd want your 
worker
service to have 
PREFECT_API_URL=<http://server:4200/api>
as it will reference the Docker service name, not the external server name.
👍 1
a
Was just confirming I'd got that right is all, nothing to do with the auth issue
b
In your 
server
service, you'd need to set 
PREFECT_API_URL=<http://exampleserver:4200/api>
as this is what your local machine's browser will use to connect to your API exposed by your Docker host. No need to set 
PREFECT_UI_URL
.
a
Thanks I've updated that, works fine without the basic auth but same error with 😞
b
In the folder where you have your 
docker-compose.yml
file can you run this command?
Copy code
docker-compose ps
I want you to take the output of the 
<something>-worker-1
line and grab the image name. Should be something like this: 
<something>-worker
Run this command:
Copy code
docker image inspect <something>-worker --format "{{.Config.Env}}"
This will dump all the ENVs set in the container image. Can you paste that result here?
a
[PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LANG=C.UTF-8 GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D PYTHON_VERSION=3.11.10 PYTHON_SHA256=07a4356e912900e61a15cb0949a06c4a05012e213ecd6b4e84d0f67aabbee372 LC_ALL=C.UTF-8]
b
Ok. Perfect. Wanted to make sure that there was no hard-coded credentials there.
✅ 1
@Ashley Kent when you access the Prefect server UI in your browser, does it prompt you to enter the basic auth credentials?
a
It does indeed. It is just my worker that doesn't work with basic auth added
b
The only way I've been able to replicate this behavior is by commenting out the 
PREFECT_API_AUTH_STRING
env on the worker.
a
I will try setting this up on a brand new server and seeing if I can recreate the issue, I appreciate you spending then time
b
If you want to check if the worker is using the correct ENV, you can run this command:
Copy code
docker inspect <something>-worker-1 --format "{{.Config.Env}}"
If that's not the case, let me know if you have better luck on the separate server.
👍 1
a
[PREFECT_API_AUTH_STRING=admin:password PREFECT_API_URL=<http://server:4200/api> PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LANG=C.UTF-8 GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D PYTHON_VERSION=3.11.10 PYTHON_SHA256=07a4356e912900e61a15cb0949a06c4a05012e213ecd6b4e84d0f67aabbee372 LC_ALL=C.UTF-8]
✅ 1
b
So, it's correctly set...
a
Frustratingly so 😅 will let you know outcomes when I test further, thanks again
Open in Slack
PreviousNext
Powered by