Great plan. You can absolutely run

prefect deploy

from GitLab CI and use a private image registry. There are two places you need to handle auth: 1) Build/push side (your GitLab CI job) 2) Pull side (your Kubernetes work pool pulling the image) Here’s what to set up. A) Update your prefect.yaml to include registry creds on push and better tagging - Keep using the docker build/push steps (they’re correct for Prefect 3.x). To push to a private registry, either: - login to the registry in CI before

prefect deploy

(simplest), or - pass credentials to the

push_docker_image

step Example adjusting your snippet to: - tag with the commit SHA and latest - push with credentials from GitLab CI variables

# Generic metadata about this project
name: flows
prefect-version: 3.0.0

build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build-image
    requires: prefect-docker>=0.4.0
    image_name: "{{ $PREFECT_IMAGE_NAME }}"
    tag: "{{ $CI_COMMIT_SHA }}"
    additional_tags:
      - "latest"
    dockerfile: auto
    platform: "linux/amd64"

push:
- prefect_docker.deployments.steps.push_docker_image:
    requires: prefect-docker>=0.4.0
    image_name: "{{ build-image.image_name }}"
    tag: "{{ build-image.tag }}"
    additional_tags: "{{ build-image.additional_tags }}"
    credentials:
      registry_url: "{{ $CI_REGISTRY }}"
      username: "{{ $CI_REGISTRY_USER }}"
      password: "{{ $CI_REGISTRY_PASSWORD }}"

pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/flows

definitions:
  tags: &common_tags
    - "eks"
  work_pool: &common_work_pool
    name: "kubernetes"
    job_variables:
      image: "{{ build-image.image }}"
      # If pulling from a private registry, make sure to set imagePullSecrets (see section C)
      # imagePullSecrets:
      #   - name: regcred

deployments:
- name: "default"
  tags: *common_tags
  schedule: null
  entrypoint: "flows/hello.py:hello"
  work_pool: *common_work_pool

- name: "arthur"
  tags: *common_tags
  schedule: null
  entrypoint: "flows/hello.py:hello"
  parameters:
    name: "Arthur"
  work_pool: *common_work_pool

Notes: -

image_name

should include your registry namespace. In GitLab CI use

$PREFECT_IMAGE_NAME=$CI_REGISTRY_IMAGE

so the final image is like

<http://registry.gitlab.com/group/project:sha|registry.gitlab.com/group/project:sha>

. - The push step accepts a

credentials

dict; using GitLab’s built-in variables keeps secrets out of the repo. B) GitLab CI job to deploy You can either log in via Docker or rely on the credentials field above. Logging in is still recommended so the build can pull any private base images. Minimal .gitlab-ci.yml example:

stages:
  - deploy

deploy:
  stage: deploy
  image: python:3.11
  services:
    - docker:dind
  variables:
    DOCKER_HOST: <tcp://docker:2375>
    DOCKER_TLS_CERTDIR: ""
    # Set image name for Prefect steps to use
    PREFECT_IMAGE_NAME: $CI_REGISTRY_IMAGE
  before_script:
    - pip install "prefect>=3" "prefect-docker>=0.4.0"
    # Login so build can pull private bases and push works even without credentials in step
    - echo "$CI_REGISTRY_PASSWORD" | docker login "$CI_REGISTRY" -u "$CI_REGISTRY_USER" --password-stdin
    # Prefect Cloud auth (set these as masked variables)
    # - export PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<account>/workspaces/<workspace>>"
    # - export PREFECT_API_KEY="$PREFECT_API_KEY"
  script:
    - prefect deploy --all

- If you prefer to avoid DinD, you can run on a Docker executor or a runner with Docker available; the Prefect Docker build step uses the local Docker daemon. - Ensure

PREFECT_API_URL

and

PREFECT_API_KEY

are set for Prefect Cloud, or the correct API for a self-hosted server. C) Let Kubernetes pull from your private registry (imagePullSecrets) Your Kubernetes work pool must be able to pull your image. Create a docker-registry secret in the namespace where your Prefect Kubernetes worker launches jobs: ```

kubectl create secret docker-registry regcred \ --docker-server="$CI_REGISTRY" \ --docker-username="$CI_REGISTRY_USER" \ --docker-password="$CI_REGISTRY_PASSWORD" \ --namespace <your-namespace>

Then reference it in your work pool’s job variables (you can keep this in prefect.yaml via the shared work_pool section or set at the work pool level in the Prefect UI):

definitions: work_pool: &common_work_pool name: "kubernetes" job_variables: image: "{{ build-image.image }}" imagePullSecrets: - name: regcred ``` This ensures Jobs launched by the Kubernetes worker can pull your private image. Alternatively, you can add the secret to the default service account in that namespace. D) One-time setup reminders - Create a Kubernetes work pool named “kubernetes” in your Prefect workspace, and run a Kubernetes worker in your cluster that points at that work pool. - Make sure your cluster can reach your registry (network, firewall). - Keep

platform: linux/amd64

if your EKS nodes are amd64; change to arm64 if your nodes are Graviton. References: - Customize job variables (Kubernetes work pools): Customize job variables - Kubernetes deployment infra guide: Kubernetes work pools - Storing flow code and project deploy overview: Store flow code Questions to tailor this for you: - Which private registry are you using (GitLab Container Registry, ECR, others)? - What namespace does your Prefect Kubernetes worker run jobs in? - Do you want to deploy on every commit, only on default branch, or on tags? I can provide a tightened .gitlab-ci.yml for your preference.

Great question — and just to confirm, when you say “privately hosted Artifactory,” are you pulling: - a Git repository hosted behind Artifactory, or - an HTTP(S) artifact (e.g., a .zip/.tar.gz) served by Artifactory? The answer changes which storage adapter to use. Here’s how Prefect 3.x thinks about it. What Prefect needs from your repo - A Python file that defines a flow (a function decorated with

@flow

), and the rest of your project files. - An entrypoint string that tells Prefect where to import the flow from, in the form

path/to/file.py:flow_function_name

relative to the repo root. - Your dependencies installed in the worker’s environment (e.g., via base image,

requirements.txt

, or

pyproject.toml

). Prefect won’t auto-install them just by pointing at your repo. A minimal example repo layout

repo-root/
  flows/
    my_flow.py          # contains `@flow def my_flow(): ...`
  my_package/
    __init__.py
    utils.py
  requirements.txt or pyproject.toml

Entrypoint you will pass to Prefect:

flows/my_flow.py:my_flow

How to reference your source in Prefect 3.x 1) If Artifactory is hosting a private Git repository (recommended) Use

GitRepository

with credentials and pass it to

Flow.from_source

.

from prefect.flows import Flow
from prefect.runner.storage import GitRepository, GitCredentials

storage = GitRepository(
    url="<https://artifactory.example.com/scm/team/my-repo.git>",
    # Provide credentials appropriate for your Artifactory (token, username/password, or SSH)
    credentials=GitCredentials(token="**"),  # or username/password, or SSH key depending on your setup
    branch="main",  # or commit_sha="..."
)

flow = Flow.from_source(storage, entrypoint="flows/my_flow.py:my_flow")

# then deploy
flow.deploy(
    name="my-deployment",
    work_pool_name="my-pool",
)

Notes: - The only “special” requirement is that the

entrypoint

resolves to a valid flow in the repo. - Use branch or commit pinning for reproducibility. - Use a Prefect Secret/Block to store tokens instead of hardcoding. 2) If Artifactory serves an HTTP(S) archive (zip/tar.gz) Use

RemoteStorage

with the URL to your artifact, then the entrypoint must match the extracted archive’s folder structure.

from prefect.flows import Flow
from prefect.runner.storage import RemoteStorage

storage = RemoteStorage(
    url="<https://artifactory.example.com/artifactory/generic/my-project/my-build-123.zip>",
    # If your endpoint requires auth, pass the relevant settings supported by your Artifactory setup
    # (e.g., headers, basic auth). We can tailor this once you confirm your auth method.
)

flow = Flow.from_source(storage, entrypoint="flows/my_flow.py:my_flow")
flow.deploy(name="my-deployment", work_pool_name="my-pool")

What exactly goes in the repo? - Your flow code (the module containing

@flow

functions). - Any local modules you import. - A dependency manifest (

requirements.txt

or

pyproject.toml

) so your worker image/environment can be built consistently. - Optional: tests, configs, etc. Prefect only cares that the entrypoint resolves at runtime. Helpful references - Flow.from_source signature in code: prefect/flows.py - Storage adapters used by from_source: prefect/runner/storage A couple of clarifying questions so I can give you the exact snippet: - Is your Artifactory endpoint a Git repo or a downloaded artifact (zip/tar)? - Which auth method do you need (SSH key, token via HTTPS, basic auth, custom headers)? - What’s the path and name of your flow function inside the repo (the entrypoint)?

Got it — code in GitLab, container images in Artifactory. What Prefect needs in your GitLab repo - A Python file that defines a flow (a function decorated with

@flow

) - An entrypoint string to that flow:

relative/path/to/file.py:flow_function_name

- Your project code and any local modules you import - A dependency manifest (

requirements.txt

or

pyproject.toml

) so your image can install them Minimal example layout

repo-root/
  flows/
    my_flow.py          # contains `@flow def my_flow(...): ...`
  my_package/
    __init__.py
    utils.py
  requirements.txt or pyproject.toml

Entrypoint you will pass to Prefect:

flows/my_flow.py:my_flow

How to pull code from GitLab (private) Option A — SSH (recommended for GitLab) - Give your Prefect worker access to a read-only deploy key and

known_hosts

for GitLab. - Use the SSH URL:

<mailto:git@gitlab.com|git@gitlab.com>:group/project.git

(or your self-hosted GitLab domain). Option B — HTTPS with token - Store a GitLab Access Token securely (e.g., as a Prefect Secret or environment variable) - Use a

GitRepository

with credentials; or rely on your worker’s git config/credential helper. Programmatic deployment with GitLab source and Artifactory image

from prefect import flow
from prefect.flows import Flow
from prefect.runner.storage import GitRepository  # storage adapter
from prefect.docker import DockerImage            # optional for more control

# 1) Point Prefect at your GitLab repo
storage = GitRepository(
    url="<mailto:git@gitlab.com|git@gitlab.com>:my-group/my-repo.git",   # or your self-hosted GitLab SSH URL
    branch="main",                               # or pin commit_sha="..."
    include_submodules=False,
)

# 2) Load the flow from the repo entrypoint
flow = Flow.from_source(storage, entrypoint="flows/my_flow.py:my_flow")

# 3) Deploy with an image that lives in Artifactory
# Make sure your environment can `docker login <http://artifactory.example.com|artifactory.example.com>`
flow.deploy(
    name="my-deployment",
    work_pool_name="my-work-pool",               # e.g., k8s, docker, process, etc.
    image="<http://artifactory.example.com/my-proj/prefect:latest|artifactory.example.com/my-proj/prefect:latest>",  # your Artifactory repo/tag
    build=True,                                  # build the image from your project
    push=True,                                   # push to Artifactory
)

Notes - Ensure your worker runtime can: - clone/pull from GitLab (SSH or HTTPS auth configured on the worker) - build and push Docker images to Artifactory (

docker login <http://artifactory.example.com|artifactory.example.com>

) - pull the final image at run time (your cluster or worker must also be able to pull from Artifactory) - Your

entrypoint

is just a path to the flow function inside the repo; Prefect will import it and run. - If you prefer the CLI, you can also clone the repo locally and run

prefect deploy

from the project root. References - Flow.from_source (source code): flows.py - Storage adapters used by from_source: runner/storage - Docker image helper: DockerImage To tailor this exactly, can you share: - The precise entrypoint (path to the file and the flow function name)? - Your preferred GitLab auth method for the worker (SSH deploy key vs HTTPS token)? - The full Artifactory image repository you want to use (hostname/repo:tag)? - Which work pool type you’re deploying to (e.g., Kubernetes, Docker, Process)?

Absolutely — you can do this entirely with

prefect.yaml

and run it from GitLab CI. Here’s a minimal, non-programmatic “Hello World” setup you can copy/paste. Files in your GitLab repo 1) flows/hello.py

from prefect import flow, get_run_logger

@flow
def hello():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello from Prefect!")

2) prefect.yaml - This is the smallest file that works when you run

prefect deploy

from CI (CI already checks out your code, so no pull/build steps are required for a basic example).

name: hello-project
prefect-version: ">=3.0.0"

deployments:
  - name: hello-deployment
    entrypoint: flows/hello.py:hello
    work_pool:
      # Use an existing work pool
      # For the simplest hello world, a Process pool is easiest
      name: my-process-pool
    tags: ["ci"]

Notes: - For Docker/Kubernetes pools, set your image under

work_pool.job_variables.image

and make sure that image exists in Artifactory:

work_pool:
      name: my-docker-pool
      job_variables:
        image: <http://artifactory.example.com/my-namespace/prefect:latest|artifactory.example.com/my-namespace/prefect:latest>

- Make sure a worker is running and attached to that work pool. 3) .gitlab-ci.yml (minimal deploy job) - This installs Prefect, authenticates via environment, then runs

prefect deploy

.

stages: [deploy]

deploy:
  image: python:3.11
  stage: deploy
  variables:
    # Set these in GitLab CI/CD variables
    PREFECT_API_URL: $PREFECT_API_URL
    PREFECT_API_KEY: $PREFECT_API_KEY
  script:
    - pip install --upgrade pip
    - pip install prefect
    - prefect deploy

If you want to use Artifactory images - Build and push the image in CI, then reference it in

prefect.yaml

under

work_pool.job_variables.image

(as shown above). - Example CI snippet (requires your runner to support Docker-in-Docker or have Docker available):

variables:
  IMAGE: "<http://artifactory.example.com/my-namespace/prefect:latest|artifactory.example.com/my-namespace/prefect:latest>"

deploy:
  image: docker:27
  stage: deploy
  services:
    - docker:27-dind
  variables:
    DOCKER_TLS_CERTDIR: ""
    PREFECT_API_URL: $PREFECT_API_URL
    PREFECT_API_KEY: $PREFECT_API_KEY
  script:
    - apk add --no-cache python3 py3-pip
    - docker login <http://artifactory.example.com|artifactory.example.com> -u "$ARTIFACTORY_USER" -p "$ARTIFACTORY_PASSWORD"
    - docker build -t "$IMAGE" .
    - docker push "$IMAGE"
    - pip install --break-system-packages prefect
    - prefect deploy

- Make sure

work_pool.name

points to your Docker/Kubernetes pool and the same

image

tag is set in

prefect.yaml

. What happens when CI runs -

prefect deploy

reads

prefect.yaml

, creates or updates the deployment with the specified entrypoint and work pool, and your running worker picks up scheduled/triggered flow runs. Helpful references - CLI deploy entry point (source): prefect/cli/deploy.py - YAML defaults and helpers: prefect/deployments/base.py - Built-in steps you can optionally use (e.g., git clone, shell): prefect/deployments/steps If you share: - Your work pool type and name (Process vs Docker/K8s) - Whether you want CI to build/push the image or just reference an existing one I can tailor the

prefect.yaml

and CI job exactly to your setup.

Yes — make the image dynamic by templating your

prefect.yaml

with an environment variable that your GitLab CI job sets at build time. Approach 1: reference the env var directly in prefect.yaml

name: hello-project
prefect-version: ">=3.0.0"

deployments:
  - name: hello
    entrypoint: flows/hello.py:hello
    work_pool:
      name: my-docker-pool
      job_variables:
        image: ${{ env.IMAGE }}     # resolved from environment at deploy time

Approach 2: define a top-level variable, then use it

name: hello-project
prefect-version: ">=3.0.0"
variables:
  image: ${{ env.IMAGE }}

deployments:
  - name: hello
    entrypoint: flows/hello.py:hello
    work_pool:
      name: my-docker-pool
      job_variables:
        image: ${{ variables.image }}

Example GitLab CI job (build, push, deploy)

stages: [deploy]

deploy:
  image: docker:27
  stage: deploy
  services:
    - docker:27-dind
  variables:
    DOCKER_TLS_CERTDIR: ""
    REGISTRY_IMAGE: "<http://artifactory.example.com/my-namespace/prefect|artifactory.example.com/my-namespace/prefect>"
    IMAGE_TAG: "${CI_COMMIT_SHORT_SHA}"
  script:
    # 1) Build & push image to Artifactory
    - docker login <http://artifactory.example.com|artifactory.example.com> -u "$ARTIFACTORY_USER" -p "$ARTIFACTORY_PASSWORD"
    - docker build -t "$REGISTRY_IMAGE:$IMAGE_TAG" .
    - docker push "$REGISTRY_IMAGE:$IMAGE_TAG"

    # 2) Set IMAGE env var for prefect.yaml templating
    - export IMAGE="$REGISTRY_IMAGE:$IMAGE_TAG"

    # 3) Install Prefect and deploy (non-interactive)
    - apk add --no-cache python3 py3-pip
    - pip install --break-system-packages --upgrade pip prefect
    - prefect deploy
  only:
    - main
  variables:
    PREFECT_API_URL: $PREFECT_API_URL
    PREFECT_API_KEY: $PREFECT_API_KEY

Hello World flow (for completeness)

# flows/hello.py
from prefect import flow, get_run_logger

@flow
def hello():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello from Prefect!")

How it works - GitLab CI builds and pushes your image with a unique tag (e.g., commit SHA). - CI exports

IMAGE=<your-registry>/<repo>:<tag>

. -

prefect deploy

reads

prefect.yaml

and substitutes

${{ env.IMAGE }}

(or

${{ variables.image }}

). - The deployment uses that image for runs in your Docker/Kubernetes work pool. Notes - Make sure a worker is running for

my-docker-pool

and can pull from Artifactory. - For a Process work pool, the

image

setting isn’t used. - Keep your Prefect credentials in GitLab CI/CD variables:

PREFECT_API_URL

,

PREFECT_API_KEY

. References - Prefect templating utilities (placeholders/env/variables): templating - Deploy CLI and YAML plumbing: deployments/base.py, cli/deploy.py

Short answer: yes. For Docker or Kubernetes work pools, the container image that runs your flow must have Prefect installed (plus your code’s dependencies). The engine that executes the flow lives inside that runtime container, so it needs Prefect available. What to include in your runtime image - Prefect (pin to a compatible 3.x version) - Your project deps (requirements.txt/pyproject.toml) - If you pull code at runtime from Git (e.g., GitLab via SSH), include git and any SSH tooling you need; ensure credentials are available at runtime Minimal Dockerfile example

FROM python:3.11-slim

# (Optional) OS deps if your project needs them
# RUN apt-get update && apt-get install -y --no-install-recommends git openssh-client && rm -rf /var/lib/apt/lists/*

# Install Prefect and your dependencies
RUN pip install --no-cache-dir "prefect>=3,<4"
# If you vendor your deps:
# COPY requirements.txt .
# RUN pip install --no-cache-dir -r requirements.txt

# (Optional) add your project files if you bake code into the image
# COPY . /app
# WORKDIR /app

Notes - If you use a Process work pool, the image setting isn’t used; Prefect just needs to be installed on the worker host. - If you’re using Git-based source at run time (Flow.from_source), the container typically needs the tools to fetch your code (e.g., git + SSH config if using SSH). - If you build images in CI, tag them uniquely (e.g., with the commit SHA), push to Artifactory, and reference the tag dynamically in

prefect.yaml

via an environment variable (as shown earlier). References - Storage adapters and runtime components: prefect/runner/storage - Docker image helper (for programmatic builds): prefect/docker/docker_image.py