<@ULVA73B9P> I'm trying to deploy a flow from my l...
# ask-marvin
k
@Marvin I'm trying to deploy a flow from my local machine to Prefect OSS on Kubernetes. I'm struggling with working directory not being found when the flow runs. Give an example of prefect.yaml to deploy to k8s
m
thought for 378.0 seconds
Got it — “working directory not found” usually means the flow run container can’t find your code at the path your
entrypoint
expects. In Prefect 3.x you can solve this in two common ways: - Build a Docker image that contains your code and set the container’s working directory - Pull your code from Git at runtime and set the working directory accordingly Below are two complete
prefect.yaml
examples for Kubernetes work pools that address the working directory. Example A: Build a Docker image with your code (recommended for production) - Your project files are copied into the image - Working directory is set to
/opt/prefect
-
entrypoint
paths are relative to
/opt/prefect
Copy code
name: my-project
prefect-version: 3.0.0

# Build a Docker image that includes your code
build:
  - prefect.deployments.steps.run_shell_script:
      id: get_sha
      script: git rev-parse --short HEAD
  - prefect_docker.deployments.steps.build_docker_image:
      id: build_image
      requires: prefect-docker>=0.4.0
      image_name: "{{ $PREFECT_IMAGE_NAME }}"
      tag: "{{ get_sha.stdout }}"
      dockerfile: auto
      # Important if you build on Apple Silicon for a Linux cluster
      platform: linux/amd64

# Push the image to your registry
push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker>=0.4.0
      image_name: "{{ build_image.image_name }}"
      tag: "{{ build_image.tag }}"

# Set the working directory for the flow run
pull:
  - prefect.deployments.steps.set_working_directory:
      directory: /opt/prefect

deployments:
  - name: production
    description: Run my flow on Kubernetes using a built image
    # Relative to /opt/prefect in the image
    entrypoint: flows/my_flow.py:my_flow

    work_pool:
      name: kubernetes
      # Optionally set a queue if you use them
      # work_queue_name: default
      job_variables:
        # Use the image built above
        image: "{{ build_image.image }}"
        namespace: prefect
        service_account_name: prefect-worker
        image_pull_policy: IfNotPresent
        # Resource requests/limits (optional but recommended)
        cpu_request: "500m"
        memory_request: "512Mi"
        cpu_limit: "1"
        memory_limit: "1Gi"
How to deploy this: - Set
PREFECT_IMAGE_NAME
to your registry/repo (e.g. `export PREFECT_IMAGE_NAME=ghcr.io/you/my-project`) - Run
prefect deploy --name production
Example B: Pull from Git at runtime (no image build) - Clones your repo when the flow run starts - Sets working directory to the clone path - Uses a stock Prefect image - Tip: use
agit_clone
so you don’t need
git
installed in the image
Copy code
name: my-project
prefect-version: 3.0.0

pull:
  # Clone your repo into /opt/prefect/app inside the container
  - prefect.deployments.steps.agit_clone:
      repository: <https://github.com/owner/repo.git>
      branch: main
      clone_directory_name: app
  # Make that the working directory
  - prefect.deployments.steps.set_working_directory:
      directory: /opt/prefect/app

deployments:
  - name: k8s-git
    # This path is now relative to /opt/prefect/app
    entrypoint: flows/my_flow.py:my_flow

    work_pool:
      name: kubernetes
      job_variables:
        image: "prefecthq/prefect:3-latest"
        namespace: prefect
        service_account_name: prefect-worker
        image_pull_policy: IfNotPresent
        cpu_request: "250m"
        memory_request: "512Mi"
Why this fixes “working directory not found” - In Example A, your code is baked into the image under
/opt/prefect
and we set the working directory to match, so
entrypoint: flows/...
resolves correctly. - In Example B, we create the directory via the clone step, then set the working directory to that location before your flow code runs. Notes and tips - The
entrypoint
must be a path that exists relative to the container’s working directory.
- If you need to mount volumes or set a container
workingDir
via Kubernetes, customize your Kubernetes work pool’s base job template (Advanced settings). Those are not standard
job_variables
out of the box; you add placeholders (e.g.,
workingDir
,
volumeMounts
,
volumes
) to the template and then pass values via
job_variables
. - Make sure your Kubernetes worker is connected to the same work pool name you specify in the deployment. If you can share which approach you prefer (bake an image vs. pull from Git), I can tailor the YAML further to your setup.
k
@Marvin can I use a specific branch from my repository?
m
thought for 195.9 seconds
Yes. In Prefect 3.x you can target a specific branch (or pin to a commit) when you pull code at runtime with the pull steps in prefect.yaml. Example: clone a specific branch with agit_clone and set the working directory
Copy code
pull:
  - prefect.deployments.steps.agit_clone:
      repository: <https://github.com/owner/repo.git>
      branch: feature/my-branch
      # Optional:
      # include_submodules: true
      # clone_directory_name: app
      # access_token: "{{ prefect.blocks.secret.github-token }}"
  - prefect.deployments.steps.set_working_directory:
      directory: /opt/prefect/app  # match clone_directory_name (default is repo name)
Pin to an exact commit
Copy code
pull:
  - prefect.deployments.steps.agit_clone:
      repository: <https://github.com/owner/repo.git>
      commit_sha: 0123abc456def...
      clone_directory_name: app
  - prefect.deployments.steps.set_working_directory:
      directory: /opt/prefect/app
Note: - Use
agit_clone
to avoid requiring git in the image;
git_clone
also supports `branch`/`commit_sha` if you prefer. - Make sure your
entrypoint
is relative to the directory set by
set_working_directory
. If you deploy from code with Flow.from_source(...) - Pass a GitRepository object to specify a branch/commit (URLs like repo.git@branch aren’t parsed for refs):
Copy code
from prefect import flow
from prefect.runner.storage import GitRepository

my_flow = flow.from_source(
    source=GitRepository(
        url="<https://github.com/owner/repo.git>",
        branch="feature/my-branch",   # or commit_sha="0123abc..."
    ),
    entrypoint="flows/my_flow.py:my_flow",
)
my_flow.deploy(name="k8s-git", work_pool_name="kubernetes")
If you’re building a Docker image instead of pulling at runtime, just check out the desired branch locally (or in CI) before running
prefect deploy
so that branch’s code is baked into the image.