Parvez Mohammed
10/30/2025, 4:09 PMMarvin
10/30/2025, 4:09 PMMarvin
10/30/2025, 4:20 PM'{{ get-soh-version.stdout }}').
- You’re also forcing a no‑cache build (nocache: True), which guarantees new layers each time.
- Docker does not garbage-collect old images/tags automatically, so they accumulate locally.
Ways to tackle it
1) Stop forcing full rebuilds unless you need them
- Remove nocache: True and let Docker layer cache work. In Prefect 3.x, the step parameter is ignore_cache (defaults to False), so just omit nocache entirely or set:
ignore_cache: false
This will dramatically reduce redundant layers and build time. Docs: prefect-docker integration
2) Keep a stable local tag, but remove the versioned tag after push
If you don’t need all historical images locally, keep just a stable tag like latest for local use and remove the versioned tag after you push.
Example steps snippet:
- prefect_docker.deployments.steps.build_docker_image:
id: build_image
requires: prefect-docker>=0.6.1
image_name: n0vum/soc-estimation_dresden-sued
tag: '{{ get-soh-version.stdout }}'
additional_tags:
- latest
dockerfile: ./Dockerfile
buildargs:
GITLAB_ACCESS_TOKEN: "{{ prefect.blocks.secret.gitlab-token }}"
network_mode: host
ignore_cache: false
- prefect_docker.deployments.steps.push_docker_image:
id: push_image
image_name: n0vum/soc-estimation_dresden-sued
tag: '{{ get-soh-version.stdout }}'
# Remove the just-built versioned tag locally to prevent piling up
- prefect_shell.commands.shell_run_command:
id: cleanup_local_versioned_tag
command: docker rmi n0vum/soc-estimation_dresden-sued:{{ get-soh-version.stdout }} || true
3) Periodic prune of unused images
If you don’t run containers from these images locally, a prune step (either in your deploy or on a schedule) keeps Docker tidy:
- Remove dangling layers:
docker image prune -f
- Remove any images not used by containers and older than 24h:
docker image prune -a -f --filter "until=24h"
You can run these with a small prefect_shell.commands.shell_run_command step.
4) Optional: prune by label (if you label your images)
If your Dockerfile adds a label (e.g., LABEL com.prefect.deployment=my-deployment), you can prune by label:
docker images --filter "label=com.prefect.deployment=my-deployment" -q | tail -n +6 | xargs -r docker rmi
(This keeps the 5 most recent; adjust to taste.)
References
- Prefect YAML / steps docs: prefect.yaml guide
- Docker integration: prefect-docker
A couple quick questions to tailor the recommendation:
- Are the “stale” ones dangling <none>:<none> images, or lots of old versioned tags (e.g., :1.2.3, :1.2.4, etc.)?
- Are you pushing to a registry as part of deploy, and do you need to keep multiple versions locally for testing?