Short answer: Use a Helm pre-install/pre-upgrade hook Job that runs

prefect deploy

from your project image. This registers/updates all deployments from your

prefect.yaml

before the worker starts, and it’s safe to run on every install/upgrade because

prefect deploy

is idempotent in Prefect 3.x. Why this way - Ensures deployments exist before workers poll - Isolated from worker startup (easier to retry/debug than an initContainer) - Idempotent: repeated runs update, not duplicate, deployments Minimal example (drop into your chart as templates/deploy-flows-job.yaml)

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "your-chart.fullname" . }}-prefect-deploy
  annotations:
    "<http://helm.sh/hook|helm.sh/hook>": pre-install,pre-upgrade
    "<http://helm.sh/hook-weight|helm.sh/hook-weight>": "-5"
    "<http://helm.sh/hook-delete-policy|helm.sh/hook-delete-policy>": before-hook-creation,hook-succeeded
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: deploy
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          # Optional: if your project lives in a subdir
          # workingDir: /app
          env:
            - name: PREFECT_API_URL
              value: "{{ .Values.prefect.apiUrl }}"   # e.g. <http://prefect-server.prefect.svc.cluster.local:4200/api>
            # Only needed for Prefect Cloud or secured server
            - name: PREFECT_API_KEY
              valueFrom:
                secretKeyRef:
                  name: prefect-api-credentials
                  key: api-key
          command:
            - /bin/sh
            - -c
            - |
              set -euo pipefail
              echo "Deploying Prefect flows from prefect.yaml..."
              prefect deploy --all --no-prompt
              echo "Done."

Notes - Your image should contain your flow code and

prefect.yaml

at the working directory. Alternatively, mount them via a ConfigMap or clone in the Job before running

prefect deploy

. - Set

PREFECT_API_URL

to your in-cluster endpoint and include the

/api

suffix (for example, `http://prefect-server.prefect.svc.cluster.local:4200/api`). For self-hosted Server without auth,

PREFECT_API_KEY

is not required. - Avoid putting

prefect deploy

in a worker initContainer; failed deploys will block worker startup and make failures harder to triage. - Common alternative: run

prefect deploy

from CI/CD on code changes. This is often preferred for auditability, but if you specifically want auto-deploy with pods, the Helm hook Job is the cleanest. Related links - Prefect Helm charts - prefect-server chart - prefect-worker chart If you share how your flow code and

prefect.yaml

are packaged (image path, working dir, any build/push steps in

prefect.yaml

), I can tailor the Job to your setup.

Thanks for the details — this helps a lot. Review of your setup - prefect.yaml - You’re following the right pattern for a Kubernetes worker where code is baked into the image: - Build/push with prefect-docker steps - Pull is a no-op (set working dir) since the worker uses the baked image - Work pool job_variables set the image from

IMAGE_NAME:IMAGE_TAG

- Minor suggestions: - Consider bumping

requires: prefect-docker>=0.7.1

in both build and push steps to use the latest step features and fixes. - Make sure

IMAGE_NAME

is set wherever you run

prefect deploy

(your YAML relies on it). - GitHub Actions - This is the recommended approach for building, pushing, and deploying on code changes. - Add

--no-prompt

to ensure non-interactive runs:

prefect deploy --all --no-prompt

- For self-hosted with basic auth,

PREFECT_API_AUTH_STRING

is correct. Ensure

PREFECT_API_URL

includes the

/api

suffix (e.g.,

http://...:4200/api

). - If your Server DB is persistent, you typically won’t need to re-deploy on pod restarts; deployments remain in the DB. The “deploy on pod startup” need usually arises when the DB is new/ephemeral. How to deploy on pod startup without build/push - There isn’t a CLI flag to skip build/push. To skip those sections you must configure them as null in the YAML that you pass to

prefect deploy

. - The clean pattern is: 1) Keep your current CI file for code changes (with build/push). 2) Add a second “runtime” prefect file for cluster bootstrap that skips build/push. 3) Run a Helm pre-install/pre-upgrade hook Job that calls

prefect deploy --all --no-prompt --prefect-file <runtime file>

. Example: runtime prefect file (no build/push)

# prefect.runtime.yaml
# Keep just enough to register/update deployments; no image build/push here
build: null
push: null

pull:
- prefect.deployments.steps.set_working_directory:
    directory: /app

definitions:
  work_pool: &common_work_pool
    name: "worker"
    job_variables:
      image: "{{ $IMAGE_NAME }}:{{ $IMAGE_TAG }}"

deployments:
- name: My Flow
  entrypoint: src/my_project/flows/my_flow.py:run_flow
  parameters: {}
  tags: []
  work_pool: *common_work_pool
  schedules: []

Helm hook Job to auto-deploy at install/upgrade

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "your-chart.fullname" . }}-prefect-deploy
  annotations:
    "<http://helm.sh/hook|helm.sh/hook>": pre-install,pre-upgrade
    "<http://helm.sh/hook-weight|helm.sh/hook-weight>": "-5"
    "<http://helm.sh/hook-delete-policy|helm.sh/hook-delete-policy>": before-hook-creation,hook-succeeded
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: deploy
        # Use either:
        # 1) Your flows image (if it contains prefect + the runtime yaml), or
        # 2) A small runner image that has prefect installed and mounts your yaml
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        env:
        - name: PREFECT_API_URL
          value: "{{ .Values.prefect.apiUrl }}"  # must end with /api
        - name: PREFECT_API_AUTH_STRING
          valueFrom:
            secretKeyRef:
              name: prefect-api-credentials
              key: api-auth-string
        - name: IMAGE_NAME
          value: "{{ .Values.flows.imageName }}"
        - name: IMAGE_TAG
          value: "{{ .Values.flows.imageTag }}"
        command:
        - /bin/sh
        - -c
        - |
          set -euo pipefail
          echo "Registering Prefect deployments (no build/push)..."
          prefect deploy --all --no-prompt --prefect-file /app/prefect.runtime.yaml
          echo "Done."
        volumeMounts:
        - name: prefect-runtime-yaml
          mountPath: /app/prefect.runtime.yaml
          subPath: prefect.runtime.yaml
      volumes:
      - name: prefect-runtime-yaml
        configMap:
          name: {{ include "your-chart.fullname" . }}-prefect-runtime-yaml

Key points

- Idempotent:

prefect deploy

is safe to run at every install/upgrade; it will create on first run and update thereafter. - Two-file pattern: - CI file (build + push + deploy) used by GitHub Actions. - Runtime file (build: null, push: null) used by the Helm hook Job. - If you prefer one file, you can also set

build: null

and

push: null

per-deployment, but that’s still a code change; there’s no CLI skip flag. - Make sure the hook Job can see the correct

prefect.runtime.yaml

(via ConfigMap or baked into the image) and that the image has

prefect

installed. - If your Postgres volume is persistent, you may not need this hook at all because deployments survive pod restarts. Links - Prefect Helm charts: prefect-helm - prefect-worker chart: prefect-worker - prefect-server chart: prefect-server If you can share how you package

prefect.runtime.yaml

(inside the image vs ConfigMap), I can tailor the hook manifest to your setup.

Short answer: Yes—handle custom blocks as part of your bootstrap process before running

prefect deploy

. Do two things: - Register your custom block types (idempotent) with

prefect blocks register

- Create/update any required block instances via a small Python script Then run

prefect deploy --all --no-prompt

. This ensures any block references in your

prefect.yaml

resolve successfully during deploy. Recommended CI sequence (GitHub Actions)

- name: Install deploy dependencies
  run: pip install prefect prefect-docker your-package-with-blocks

- name: Register custom block types
  env:
    PREFECT_API_URL: ${{ vars.PREFECT_API_URL }}      # must end with /api
    PREFECT_API_AUTH_STRING: ${{ secrets.PREFECT_API_AUTH_STRING }}
  run: |
    # If your blocks live in a module:
    prefect blocks register -m your_package.blocks
    # Or if they live in a file:
    # prefect blocks register -f src/your_package/blocks.py

- name: Seed/update block instances
  env:
    PREFECT_API_URL: ${{ vars.PREFECT_API_URL }}
    PREFECT_API_AUTH_STRING: ${{ secrets.PREFECT_API_AUTH_STRING }}
    MY_API_KEY: ${{ secrets.MY_API_KEY }}   # example secret for a block
  run: python scripts/seed_blocks.py

- name: Build, push image and deploy Prefect flows
  env:
    IMAGE_TAG: ${{ needs.details.outputs.new_version }}
    PREFECT_API_URL: ${{ vars.PREFECT_API_URL }}
    PREFECT_API_AUTH_STRING: ${{ secrets.PREFECT_API_AUTH_STRING }}
  run: prefect deploy --all --no-prompt

Example seed script for block instances

# scripts/seed_blocks.py
import os
from pydantic import SecretStr
# Replace with your actual Block class import(s)
from your_package.blocks import MyAPI  # subclass of prefect.blocks.core.Block

def main():
    api_key = os.environ["MY_API_KEY"]
    block = MyAPI(api_key=SecretStr(api_key), base_url=os.environ.get("MY_API_BASE_URL", "<https://api.example.com>"))
    # Ensure a predictable slug that your prefect.yaml can reference
    block.save("my-api", overwrite=True)

if __name__ == "__main__":
    main()

Notes and best practices - Order matters: register block types and create/update instances before

prefect deploy

so any

{{ prefect.blocks.* }}

references in

prefect.yaml

resolve. - Idempotency:

prefect blocks register

updates existing type registrations;

.save(..., overwrite=True)

updates instances—safe to re-run on every pipeline. - Packages: Make sure your CI environment installs the package(s) that define your custom blocks (and any collections like

prefect-aws

) before

prefect blocks register

. - Auth and URL: Keep using

PREFECT_API_AUTH_STRING

for self-hosted basic auth and ensure

PREFECT_API_URL

ends with

/api

. If you prefer “deploy on pod startup,” you can also run the same two steps (register + seed instances) in a Helm pre-install/pre-upgrade hook Job immediately before

prefect deploy --prefect-file prefect.runtime.yaml --no-prompt

. For secrets hygiene, CI is generally the better place to create/update block instances unless you’re comfortable injecting Kubernetes secrets into the hook Job. Helm charts for reference: - prefect-worker: prefect-worker - prefect-server: prefect-server If you share where your custom block classes live (module/file) and which instances you need, I can provide a tailored register command and seeding script.

You’re right — there isn’t a deploy-specific --no-prompt flag. To run prefect deploy non-interactively in Prefect 3.x, use the global CLI prompt toggle instead: - CLI: prefix your command with

prefect --prompt=false ...

- Or set the setting:

PREFECT_CLI_PROMPT=false

Examples - With prefect.yaml:

prefect --prompt=false deploy --all

- Without prefect.yaml (explicit fields to avoid prompts):

prefect --prompt=false deploy ./path/to/flow.py:flow_fn --name my-deployment --pool my-pool

Tips to avoid any prompts - Ensure every deployment in your prefect.yaml has: - a name - an entrypoint - a work pool name (and any required job_variables) - If you don’t want build/push to run in certain contexts (e.g., Helm bootstrap), set them to null in the YAML you pass via

--prefect-file

, since there are no skip flags:

build: null
  push: null

Thanks for catching that. Let me know if you want me to adjust the earlier examples to use

--prompt=false

in your CI and Helm hook snippets.

You’re absolutely right — thank you for the correction. The non-interactive toggle is a global root option, so it must come immediately after prefect: - Correct: -

prefect --no-prompt deploy --all

-

prefect --no-prompt deploy -n my-deployment

- Also valid (explicit boolean): -

prefect --prompt=false deploy --all

- Equivalent via env: -

PREFECT_CLI_PROMPT=false prefect deploy --all

I’ll use

prefect --no-prompt ...

in examples going forward.

You’re hitting a Kubernetes RBAC + default namespace mismatch. The Kubernetes worker uses the “namespace” from the job variables; if it’s not set, it defaults to “default”. Your worker’s service account lives in “bgriffiths”, so it can’t create Jobs in “default”. Set the namespace explicitly so jobs are created in “bgriffiths”. You have a few good options: Option A — Set per-deployment in your prefect.yaml (quick, recommended) Add namespace (and optionally the service account) to your common work_pool job_variables so every deployment uses it by default:

definitions:
  work_pool: &common_work_pool
    name: "worker"
    job_variables:
      image: "{{ $IMAGE_NAME }}:{{ $IMAGE_TAG }}"
      namespace: "bgriffiths"
      service_account_name: "prefect-worker"

Then re-run your deploy:

prefect --no-prompt deploy --all

Option B — Set a default on the work pool (applies to all deployments that don’t override it) - In the UI: Work Pools -> your Kubernetes pool -> Settings -> Default job variables -> set Namespace to “bgriffiths” (and Service Account Name if needed). - Or via CLI by updating the pool’s base job template (more work): 1) Export the default template:

prefect work-pool get-default-base-job-template --type kubernetes --file kubepool.json

2) Edit kubepool.json to set the default for the namespace variable (look for

variables.properties.namespace.default

and set to

"bgriffiths"

). 3) Update the pool:

prefect work-pool update "worker" --base-job-template kubepool.json

Option C — Fix RBAC instead (only if you truly want jobs in “default”) Grant your worker’s service account permissions to create Jobs in the “default” namespace (Role + RoleBinding in “default”, or a ClusterRoleBinding). Most teams prefer running jobs in the same namespace as the worker for least-privilege. Notes - You can still override namespace per deployment with

work_pool.job_variables.namespace

if needed. - Ensure the

service_account_name

you set exists in the target namespace and has Role/RoleBinding to create Jobs, watch Pods, and read Pod logs. If you share how you’re creating the work pool (via UI, CLI, or Helm automation), I can provide exact commands/manifests to set the default namespace at creation time.