oleander-whirlwind
09/30/2025, 11:49 AMMarvin
09/30/2025, 11:49 AMMarvin
09/30/2025, 11:59 AM.deploy(...) must be authenticated to your Artifactory registry (e.g., via docker login or CI secrets).
- For workers to pull at run time: configure your work pool/worker with Docker registry credentials (via a Prefect block) or Kubernetes image pull secrets.
1) Point .deploy() at Artifactory
Pass a fully-qualified image name (including your Artifactory host) to Flow.deploy. You can use either a string or DockerImage for extra build kwargs.
Example:
from prefect import flow
from prefect.docker import DockerImage
@flow
def my_flow():
...
# Source could be Git, local path, etc.
flow.from_source(
source="<https://github.com/acme/prefect-repo>",
entrypoint="flows/my_flow.py:my_flow",
).deploy(
name="prod",
work_pool_name="docker-pool",
image=DockerImage(
name="<http://artifactory.myco.com/prefect/my-flow|artifactory.myco.com/prefect/my-flow>", # <- your Artifactory host/repo
tag="2025-09-30",
dockerfile="Dockerfile",
# Optional: pass Docker build args (e.g., for Artifactory PyPI mirrors, proxies, etc.)
buildargs={
"PIP_INDEX_URL": "<https://artifactory.myco.com/api/pypi/pypi/simple>"
},
),
build=True,
push=True,
)
Notes:
- Make sure the machine running this code can push (e.g., docker login <http://artifactory.myco.com|artifactory.myco.com>).
- If your Artifactory proxies Docker Hub and your company requires all pulls go through Artifactory, update your Dockerfile base image to use the proxied path, e.g.:
- Instead of FROM python:3.11-slim, use FROM <http://artifactory.myco.com/docker-hub-proxy/library/python:3.11-slim|artifactory.myco.com/docker-hub-proxy/library/python:3.11-slim>
- Or configure your Docker daemon with a registry mirror that points to Artifactory (outside Prefect).
2) Authenticating the push (during deploy)
Flow.deploy’s push step uses your local Docker credentials. Options:
- Local: run docker login <http://artifactory.myco.com|artifactory.myco.com> before .deploy(...).
- CI: set DOCKER_AUTH_CONFIG or run a docker login step with CI secrets before invoking your deploy script.
- If you’re using prefect.yaml with steps, you can explicitly call the push step with credentials:
steps:
- prefect_docker.deployments.steps.build_docker_image:
image_name: <http://artifactory.myco.com/prefect/my-flow|artifactory.myco.com/prefect/my-flow>
tag: "{{ version }}"
- prefect_docker.deployments.steps.push_docker_image:
image_name: <http://artifactory.myco.com/prefect/my-flow|artifactory.myco.com/prefect/my-flow>
tag: "{{ version }}"
credentials:
username: "{{ prefect.variables.ARTIFACTORY_USER }}"
password: "{{ prefect.variables.ARTIFACTORY_TOKEN }}"
registry_url: "<http://artifactory.myco.com|artifactory.myco.com>"
See the step signatures here:
- prefect-docker deployment steps
3) Authenticating workers for pulls (runtime)
- Docker worker:
- Create a DockerRegistryCredentials block and use it in your Docker work pool’s base job template (or per-deployment).
- Example to create/save a credentials block:
from prefect_docker.credentials import DockerRegistryCredentials
creds = DockerRegistryCredentials(
username="svc-prefect",
password="***", # load from env/secret manager
registry_url="<http://artifactory.myco.com|artifactory.myco.com>",
)
creds.save("my-artifactory-creds", overwrite=True)
- In your Docker work pool’s base job template, reference it:
{
"image": "{{ image }}",
"env": "{{ env }}",
"command": "{{ command }}",
"registry_credentials": "{{ block('docker-registry-credentials', 'my-artifactory-creds') }}"
}Marvin
09/30/2025, 11:59 AM.deploy(...) locally or in CI?
- Do you already have an Artifactory-proxied path for your base images (FROM lines), or do you rely on a registry mirror in Docker?
Happy to provide a concrete example (prefect.yaml or Python) once I know your worker type and where you run deploy.
Helpful links
- Docker integration docs: prefect-docker
- Docker worker docs: Docker worker
- Kubernetes integration: prefect-kubernetesoleander-whirlwind
10/07/2025, 6:28 AMMarvin
10/07/2025, 6:28 AMMarvin
10/07/2025, 6:30 AMDockerImage.build()).
- During the build, Docker needs to pull a base image from Docker Hub (registry-1.docker.io).
- The Docker daemon can’t maintain the TLS connection to Docker Hub; the peer (or something on the network path) is resetting it. This is a network/connectivity issue, not a Prefect bug.
Likely causes:
- Corporate proxy or firewall intercepting/terminating TLS.
- Docker Hub being blocked or rate-limited for anonymous pulls.
- Intermittent network issues or DNS problems.
- TLS inspection or MITM box that Docker doesn’t trust.
- Docker daemon not configured for proxies or mirrors.
Quick checks
1) Can the host reach Docker Hub?
curl -v <https://registry-1.docker.io/v2/>
You should see a 401 Unauthorized (that’s expected) — if it fails, it’s a network/proxy issue.
2) Can Docker pull any image?
docker pull alpine:3.20
docker pull hello-world
3) Are you logged in to Docker Hub (to avoid rate limits)?
docker login
Fixes and workarounds
- If you’re behind a proxy or corporate firewall:
- Configure the Docker daemon to use your HTTP/HTTPS proxy and set NO_PROXY for internal addresses:
- Linux/systemd: Configure Docker daemon proxy (systemd)
- Build-time proxy for apt/pip in Dockerfile: Build-time proxies
- Allowlist these hosts/ports if needed: registry-1.docker.io:443, auth.docker.io:443.
- Use a Docker registry mirror (pull-through cache) to avoid hitting Docker Hub directly:
- Configure "registry-mirrors" in `/etc/docker/daemon.json`: Docker daemon configuration (registry-mirrors)
- Set up or use an existing mirror: Use a registry mirror / pull-through cache
- Pre-pull the base image to prime the Docker cache:
- Example:
docker pull prefecthq/prefect:3-python3.12
Replace with the exact tag you use. Once it’s local, docker build won’t need to reach Docker Hub for the base layer unless you force pulls.
- Use a custom Dockerfile that points at an internal/mirrored base image instead of Docker Hub:
- Dockerfile example:
FROM <http://registry.internal.example.com/prefect/prefect:3-python3.12|registry.internal.example.com/prefect/prefect:3-python3.12>
# your additional steps...
- Then tell Prefect to use this file when deploying:
from prefect import flow
from prefect.docker import DockerImage
image = DockerImage(
name="<http://registry.internal.example.com/your-team/my-flow|registry.internal.example.com/your-team/my-flow>",
tag="2025-10-07",
dockerfile="Dockerfile",
# Optionally help Docker during build if needed:
# build_kwargs={"network_mode": "host"}
)
my_flow.deploy(
name="my-deployment",
image=image,
push=False, # still builds locally, doesn’t push
)
- If you need to pass proxy env vars during build steps (apt/pip), you can do:
image = DockerImage(
name="<http://registry.internal.example.com/your-team/my-flow|registry.internal.example.com/your-team/my-flow>",
dockerfile="Dockerfile",
build_kwargs={
"buildargs": {
"HTTP_PROXY": "<http://proxy:3128>",
"HTTPS_PROXY": "<http://proxy:3128>",
"NO_PROXY": "localhost,127.0.0.1,.<http://internal.example.com|internal.example.com>"
}
},
)
Note: build args won’t affect pulling the base image, but they help in RUN steps.Marvin
10/07/2025, 6:30 AMnetwork_mode=host for the build as a test:
build_kwargs={"network_mode": "host"}
- Check Docker Hub status if this might be transient: Docker Hub status
Notes about Prefect 3
- You’re on Prefect 3.x, and using my_flow.deploy(...) which can build images via prefect.docker.DockerImage. The failure is occurring before Prefect can finish the deployment because Docker cannot pull the base image.
- If you don’t need a container image for this deployment, you can temporarily omit the image= argument to bypass the Docker build and run with a non-container infrastructure until network issues are resolved.
A few questions to help me tailor the fix:
- Are you behind a corporate proxy/VPN or on a locked-down network?
- Can you successfully run docker pull alpine:3.20 from this same host?
- Are you using Prefect’s auto-generated Dockerfile or a custom one?
- Which OS and Docker engine version are you running?
If you share those details, I can suggest the most direct config changes for your environment.oleander-whirlwind
10/07/2025, 6:34 AMMarvin
10/07/2025, 6:34 AMMarvin
10/07/2025, 6:35 AMlocal-docker_client_kas/prefect/numpy-mul, Prefect tried to build anyway because Flow.deploy() defaults to build=True, which triggers a Docker build (and pulls a base image from Docker Hub).
Use the image directly and skip the build:
- Set image to your existing image name
- Set build=False (and keep push=False)
Example:
from prefect import flow
@flow
def my_flow():
...
my_flow.deploy(
name="my-deployment",
work_pool_name="your-docker-work-pool",
image="local-docker_client_kas/prefect/numpy-mul:latest", # or your specific tag
build=False, # <- prevents Prefect from building (no Docker Hub pull)
push=False, # <- prevents push
)
Notes:
- Make sure the Docker worker that will run this deployment can pull/use that image name. If the worker runs on the same machine and the image is local, great. If local-docker_client_kas is a private registry hostname, ensure the worker host is logged in (docker login <registry>) and trusts the registry (add CA or mark as insecure if needed).
- If local-docker_client_kas is a Docker context name (not a registry), Prefect will still use the worker’s Docker daemon, not your local context. In that case, either run the worker against that same daemon or push the image to a registry accessible to the worker.
If you can share:
- whether local-docker_client_kas is a registry or a Docker context
- where your Docker worker is running (same host vs remote)
I can tailor the exact configuration steps.oleander-whirlwind
10/07/2025, 6:49 AMMarvin
10/07/2025, 6:49 AMMarvin
10/07/2025, 6:52 AMprefect CLI inside the image.
- Your image local-docker_client_kas/prefect/numpy-mul doesn’t have the prefect CLI available (or an ENTRYPOINT is interfering).
Quick verification
- From the worker host, try:
docker run --rm local-docker_client_kas/prefect/numpy-mul:latest prefect version
docker run --rm local-docker_client_kas/prefect/numpy-mul:latest which prefect
If these fail, the image doesn’t include Prefect.
Two common fixes
1) Base your image on Prefect (recommended)
- Rebuild your image FROM a Prefect base, or extend your existing image and install Prefect:
# Option A: start from Prefect base
FROM prefecthq/prefect:3-python3.12
# add your code/deps here...
# Option B: extend your image and add Prefect
FROM local-docker_client_kas/prefect/numpy-mul:latest
RUN python -m pip install --no-cache-dir "prefect>=3,<4"
- Use that tag in your deployment, with build/push disabled:
my_flow.deploy(
name="my-deployment",
work_pool_name="your-docker-work-pool",
image="local-docker_client_kas/prefect/numpy-mul:with-prefect",
build=False,
push=False,
)
2) If your image has a custom ENTRYPOINT
- Prefect sets the container command to prefect .... If your image’s ENTRYPOINT wraps/overrides the command, you can clear it at runtime:
my_flow.deploy(
name="my-deployment",
work_pool_name="your-docker-work-pool",
image="local-docker_client_kas/prefect/numpy-mul:latest",
build=False,
push=False,
job_variables={
"container_create_kwargs": {"entrypoint": []}
},
)
This only helps if prefect is actually installed in the image; otherwise you’ll see the same error.
Also confirm
- The worker is running on the same Docker host that has the image, or the image name is pullable from that host. You can enforce no pulls:
job_variables={"image_pull_policy": "Never"}
- If local-docker_client_kas is a private registry, ensure the worker host is logged in: docker login local-docker_client_kas.
If you can share:
- Does your image currently include Prefect?
- Does it set an ENTRYPOINT?
- Is the worker on the same machine as the image?
I’ll suggest the exact minimal change for your setup.oleander-whirlwind
10/07/2025, 6:54 AMMarvin
10/07/2025, 6:54 AMMarvin
10/07/2025, 6:54 AMprefect CLI inside the container. If prefect isn’t on PATH in the image, you’ll get the “executable file not found” error you saw.
Ways to satisfy this:
- Base from a Prefect image:
FROM prefecthq/prefect:3-python3.12
# add your code/deps...
- Or install Prefect into your existing image:
FROM local-docker_client_kas/prefect/numpy-mul:latest
RUN python -m pip install --no-cache-dir "prefect>=3,<4"
- Verify:
docker run --rm your-image prefect version
Only exception: if you don’t want Prefect in a container at all, use a non-container worker (e.g., Process worker) so Prefect runs from the host environment instead of inside an image.
If your image has a custom ENTRYPOINT, make sure it doesn’t block the worker’s command, or clear it via deployment `job_variables`:
job_variables={"container_create_kwargs": {"entrypoint": []}}
(This still requires Prefect to be installed in the image.)oleander-whirlwind
10/07/2025, 6:58 AMMarvin
10/07/2025, 6:58 AMMarvin
10/07/2025, 6:59 AMprefect ... inside it.
- ENTRYPOINT notes:
- Prefer not to override ENTRYPOINT. Let Prefect set the command.
- If you do have a custom ENTRYPOINT, make sure it ultimately execs the command passed by the worker (which will be the prefect ... invocation). A safe pattern:
# Dockerfile
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
# entrypoint.sh
#!/bin/sh
set -e
# your init steps here
exec "$@" # <- runs the CMD passed by the worker, e.g. the Prefect command
- If your image’s ENTRYPOINT conflicts, you can clear it at deploy time:
job_variables={"container_create_kwargs": {"entrypoint": []}}
(You still need Prefect installed in the image.)
2) Workload containers you launch from your flow (docker “subprocess” calls)
- These do NOT need Prefect. They’re just containers you start from your flow code (e.g., with the Docker SDK or subprocess).
- Your flow still runs inside the orchestration container (which needs Prefect), but the child containers can be any image with its own ENTRYPOINT/CMD.
Example: run a child container from a Prefect flow running on a Docker worker
from prefect import flow, task
import docker
@task
def run_child_container():
client = docker.from_env()
# Runs your image with its own entrypoint/CMD
logs = client.containers.run(
"local-docker_client_kas/prefect/numpy-mul:latest",
detach=False,
remove=True,
environment={"FOO": "bar"},
)
print(logs.decode() if isinstance(logs, bytes) else logs)
@flow
def my_flow():
run_child_container()
# Deploy using a Prefect-enabled image for the ORCHESTRATION container
my_flow.deploy(
name="my-deployment",
work_pool_name="your-docker-work-pool",
image="prefecthq/prefect:3-python3.12",
build=False,
push=False,
job_variables={
# allow the flow to talk to the host Docker daemon
"volumes": ["/var/run/docker.sock:/var/run/docker.sock"],
# optionally set DOCKER_HOST if needed
"env": {"DOCKER_HOST": "unix:///var/run/docker.sock"},
},
)
Key takeaways
- Orchestration container (the one the worker starts): Prefect required.
- Child containers you start from your flow: Prefect not required; your own ENTRYPOINT is fine.
- If you have a custom ENTRYPOINT in the orchestration image, ensure it execs the CMD the worker supplies (or clear it via container_create_kwargs.entrypoint=[]).oleander-whirlwind
10/07/2025, 7:16 AMMarvin
10/07/2025, 7:16 AMMarvin
10/07/2025, 7:17 AMdocker ps -a --no-trunc --filter "status=exited" --format "{{.ID}}\t{{.Names}}\t{{.Status}}"
- Inspect and view logs:
docker inspect <container_id> --format '{{.Config.Entrypoint}} {{.Config.Cmd}}'
docker logs <container_id>
These logs will show why it exited with code 1.
2) Ensure Prefect logs are streamed:
- Deployment/job variables can set this; it defaults to True for Docker worker:
job_variables={"stream_output": True}
If the container dies before Prefect starts, you may still only see infra (docker) logs, so the host docker logs is the best source.
Quick sanity checks
- Verify Prefect is in the image and runnable:
docker run --rm YOUR_IMAGE prefect version
docker run --rm --entrypoint '' YOUR_IMAGE prefect version # bypass ENTRYPOINT if needed
- Verify your ENTRYPOINT script (if any) ends with:
exec "$@"
so that Prefect’s command provided by the worker actually runs.
- See exactly what the worker is trying to run by starting it with debug logs:
PREFECT_LOGGING_LEVEL=DEBUG prefect worker start -p your-docker-work-pool
This will show the container create/start parameters.
If you share the output of:
- docker inspect <container_id> --format '{{.Config.Entrypoint}} {{.Config.Cmd}}'
- docker logs <container_id>
I can pinpoint the exact fix (ENTRYPOINT adjustment, install Prefect, dependency issue, etc.).oleander-whirlwind
10/07/2025, 7:23 AMMarvin
10/07/2025, 7:23 AMMarvin
10/07/2025, 7:23 AMprefect ... invocation at runtime. It does not require an ENTRYPOINT.
- If you do have an ENTRYPOINT, it must pass through the command Prefect supplies. Use an exec pattern.
Good patterns
- No ENTRYPOINT in your Dockerfile:
# No ENTRYPOINT; Prefect supplies the command at runtime
- Pass-through ENTRYPOINT:
# Dockerfile
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
# entrypoint.sh
#!/bin/sh
set -e
# optional init work...
exec "$@" # critical: runs the Prefect command that the worker passes
Resetting/clearing ENTRYPOINT at deploy time
- If your image has an ENTRYPOINT you want to ignore for Prefect runs, clear it in your deployment:
my_flow.deploy(
name="my-deployment",
work_pool_name="your-docker-work-pool",
image="your/image:tag",
build=False,
push=False,
job_variables={
"container_create_kwargs": {
"entrypoint": [] # clears ENTRYPOINT for this run
}
},
)
About ENTRYPOINT [""]
- In a Dockerfile, ENTRYPOINT [""] is not valid and will fail.
- At runtime, tools may use an empty value to clear the entrypoint; with Prefect we recommend the explicit array form entrypoint: [] via container_create_kwargs.
Rule of thumb
- Prefer no ENTRYPOINT or a pass-through one that ends with exec "$@".
- Ensure the image has the prefect CLI installed; otherwise the worker’s command will fail even with a perfect ENTRYPOINT. Verify with:
docker run --rm your/image:tag prefect versionoleander-whirlwind
10/07/2025, 7:29 AMMarvin
10/07/2025, 7:29 AMMarvin
10/07/2025, 7:29 AMdocker run --rm your-image prefect version
2) ENTRYPOINT is optional, but if present it must be compatible
- You do not need to set an ENTRYPOINT in your Dockerfile. The simplest and most reliable setup is: no ENTRYPOINT, Prefect installed, and let the worker provide the command.
- If you do have an ENTRYPOINT, it must pass through the command Prefect supplies. In practice, your entrypoint script should end with:
exec "$@"
so the Prefect command (the CMD) actually runs.
- If your ENTRYPOINT ignores or overrides the CMD, the container can exit with code 1 before Prefect starts.
Putting it together:
- Correct statement: You don’t need to define an ENTRYPOINT at all. If you choose to define one, it has to be a pass-through entrypoint that ultimately execs the CMD the worker provides.
- The earlier “wrong entrypoint” comment means: if you have an ENTRYPOINT and it doesn’t exec "$@", it can cause the “exited with status 1” behavior. That’s separate from the “prefect not found” error, which is solved by installing Prefect in the image.
Helpful knobs during deployment:
- To ignore a problematic entrypoint at runtime:
job_variables={"container_create_kwargs": {"entrypoint": []}}
- To verify what the worker is doing, start it with debug logs:
PREFECT_LOGGING_LEVEL=DEBUG prefect worker start -p your-docker-work-pool
If you share your current Dockerfile’s ENTRYPOINT (if any) and the result of docker run --rm your-image prefect version, I can give you the exact minimal change.oleander-whirlwind
10/07/2025, 9:43 AMMarvin
10/07/2025, 9:43 AMMarvin
10/07/2025, 9:44 AM# Start your worker with a resolvable URL
PREFECT_API_URL=http://<server-host-or-ip>:4200/api prefect worker start -p your-docker-work-pool
- Or set it per deployment:
my_flow.deploy(
...,
job_variables={
"env": {"PREFECT_API_URL": "http://<server-host-or-ip>:4200/api"}
},
)
2) Put the job container on the same Docker network as the Prefect Server
- Find the server container’s network(s):
docker inspect <prefect-server-container-name> --format '{{json .NetworkSettings.Networks}}' | jq .
- Add that network to your deployment so the job can resolve prefect-server:
my_flow.deploy(
...,
job_variables={
"networks": ["<the-server-network-name>"]
},
)
3) If you run the worker in the same compose stack/network as the server
- You can keep using the hostname prefect-server, but ensure your worker submits jobs that join that same network (see option 2).
Quick tests
- From the worker host, check that the API is reachable by the address you plan to use:
curl -sS http://<server-host-or-ip>:4200/api/health
- From a test container on the intended network:
docker run --rm --network <the-server-network-name> curlimages/curl -sS <http://prefect-server:4200/api/health>
Once PREFECT_API_URL resolves from inside the flow-run container, the “infrastructure exited with non-zero status code 1” should go away.
If you can share:
- How your Prefect Server is running (docker compose? container name/network?)
- How you start the worker (any PREFECT_API_URL set?)
I can give the exact command and network name to use.oleander-whirlwind
10/07/2025, 9:51 AMMarvin
10/07/2025, 9:51 AMMarvin
10/07/2025, 9:54 AMFROM prefecthq/prefect:3-python3.12
WORKDIR /app
COPY . /app
# install your deps if needed
# RUN pip install -r requirements.txt
- Build/tag/push or make available locally, then deploy using that image and disable build/push in Prefect:
my_flow.deploy(
name="image-average",
work_pool_name="your-docker-work-pool",
image="your-registry/your-image:with-code",
build=False,
push=False,
)
- Ensure your deployment’s entrypoint matches the path inside the image, e.g. deployments/local_deployment_image_average.py:cluster_image_and_process.
Option B: Mount your code into the container at runtime
- Keep your existing image, and mount the project directory into /app with job_variables.volumes:
my_flow.deploy(
name="image-average",
work_pool_name="your-docker-work-pool",
image="your-registry/your-image:tag",
build=False,
push=False,
job_variables={
"volumes": [
"/abs/path/on/worker/host/project:/app" # must exist on the worker host
],
# optional: set working dir if needed
"container_create_kwargs": {"working_dir": "/app"}
},
)
- The path on the left must exist on the worker’s machine (not your laptop if the worker is remote).
Option C: Use Flow.from_source with a remote source (recommended for portability)
- Put your code in a Git repo or package and let Prefect pull it at runtime:
from prefect import flow
flow = flow.from_source(
source="<https://github.com/your-org/your-repo>",
entrypoint="deployments/local_deployment_image_average.py:cluster_image_and_process",
)
flow.deploy(
name="image-average",
work_pool_name="your-docker-work-pool",
image="prefecthq/prefect:3-python3.12",
build=False,
push=False,
)
- You can also point source to a local path that’s accessible to the worker, but remote git is best for distributed workers.
Sanity checks
- Verify where Prefect expects the file: it uses the entrypoint path relative to the working directory inside the container. With WORKDIR /app and COPY . /app, the file should be at /app/deployments/local_deployment_image_average.py.
- Confirm the file exists by running the image interactively:
docker run --rm -it your-image:with-code ls -l /app/deployments
Given your log, the quickest fix is Option A (COPY your code) or Option B (mount your project) depending on whether you want immutable images or live-mount for development.oleander-whirlwind
10/07/2025, 11:04 AMMarvin
10/07/2025, 11:04 AMMarvin
10/07/2025, 11:04 AMlog_prints=True on your flow and any tasks that call `print`:
from prefect import flow, task
@task(log_prints=True)
def my_task():
print("hello from task")
@flow(log_prints=True)
def my_flow():
print("hello from flow")
my_task()
- Or set it globally via config/env:
# in code
from prefect.settings import PREFECT_LOGGING_LOG_PRINTS, temporary_settings
# or set env var before starting worker/run: PREFECT_LOGGING_LOG_PRINTS=true
- Ensure the Docker worker streams container stdout/stderr
- It’s on by default, but if you’ve overridden it, re-enable:
my_flow.deploy(
...,
job_variables={"stream_output": True}
)
- Without streaming, raw stdout won’t be forwarded to the UI.
- Avoid output buffering
- Python may buffer prints in containers. Either:
- Use print(..., flush=True), or
- Set PYTHONUNBUFFERED=1 in the container env:
my_flow.deploy(
...,
job_variables={"env": {"PYTHONUNBUFFERED": "1"}}
)
What to expect:
- With log_prints=True, print output appears as Prefect logs under the corresponding flow/task runs.
- With only stream_output=True (and no log_prints), you may see raw container output attached to the flow run logs, but not attributed to specific tasks.
- If you see “Finished in state Completed()” but still no prints, check the container logs directly to confirm the prints are happening:
docker logs <container_id>
If you share how your flow/tasks are defined and your current deployment’s job_variables, I can suggest the minimal change to surface your prints in the UI.oleander-whirlwind
10/07/2025, 11:10 AMMarvin
10/07/2025, 11:10 AMMarvin
10/07/2025, 11:10 AM<http://infra-repos.ohb-system.de:443/local-docker_client_kas/prefect/image_average:0.0.3|infra-repos.ohb-system.de:443/local-docker_client_kas/prefect/image_average:0.0.3>
- PREFECT_API_URL: don’t use localhost inside the container unless your server is actually on the same container namespace. Point to something the container can reach:
- If your server is a container on a compose network: <http://prefect-server:4200/api> and add that network to the job
- Or use the host IP/hostname: http://<host-ip>:4200/api
- On Docker Desktop, you can use <http://host.docker.internal:4200/api>
- Env values must be strings. Use "true" for booleans.
- Keep stream_output=True (it’s default, but fine to be explicit).
- Optionally also set log_prints=True on your flow/tasks for attribution.
Corrected example:
cluster_image_and_process.deploy(
name="my-seventh-docker-deployment",
work_pool_name="docker-worker-pool-in-compose-on-brekasdev06",
image="<http://infra-repos.ohb-system.de:443/local-docker_client_kas/prefect/image_average:0.0.3|infra-repos.ohb-system.de:443/local-docker_client_kas/prefect/image_average:0.0.3>",
build=False,
push=False,
job_variables={
"env": {
"PREFECT_API_URL": "<http://prefect-server:4200/api>", # or http://<host-ip>:4200/api
"PYTHONUNBUFFERED": "1",
"PREFECT_LOGGING_LOG_PRINTS": "true"
},
"stream_output": True,
# If your Prefect Server runs on a compose network, add it here:
# "networks": ["<server_network_name>"]
},
)
And in code (recommended for clarity):
from prefect import flow, task
@task(log_prints=True)
def process_kernel():
print("hello from task")
@flow(log_prints=True)
def cluster_image_and_process():
print("hello from flow")
process_kernel()
If prints still don’t show:
- Confirm the container can reach the API (inside container): curl -sS $PREFECT_API_URL/health
- Check the container logs directly on the worker host: docker logs <container_id> to confirm the prints are emitted
- Start the worker with debug logs to verify env was passed:
- PREFECT_LOGGING_LEVEL=DEBUG prefect worker start -p docker-worker-pool-in-compose-on-brekasdev06
If you share how your Prefect Server is hosted (compose service name/network) I can suggest the exact PREFECT_API_URL and networks value.