https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> How can I use the Kubernetes decorato...
# ask-marvin
n
@Marvin How can I use the Kubernetes decorator to run a flow in an already existing docker image on ECR? It already contains all necessary dependencies
m
✅ thought for 114.0 seconds
Yes — you can point the Kubernetes decorator at an existing ECR image by setting the 
image
job variable. Minimal example:
Copy code
python
from prefect import flow
from prefect_kubernetes.decorators import kubernetes

@kubernetes(
    work_pool="my-k8s-pool",
    image="<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0>",
    image_pull_policy="IfNotPresent",
    namespace="prefect"
)
@flow
def my_flow():
    print("running in my existing ECR image")

if __name__ == "__main__":
    my_flow()
A few important notes: * 
image
should be the full ECR image URI * your image still needs Prefect installed, since Prefect will execute the flow inside that container * your Kubernetes cluster must be able to pull from ECR For private ECR images, the Kubernetes job usually also needs 
imagePullSecrets
configured in the work pool's base job template. That part is not typically set directly on the decorator — it’s usually configured on the Kubernetes work pool. Typical pattern: 1. Create a Kubernetes secret for ECR auth 2. Add 
imagePullSecrets
to the work pool's base job template 3. Use 
@kubernetes(..., image="...")
on the flow If you're deploying with 
prefect.yaml
, the equivalent is:
Copy code
yaml
deployments:
  - name: my-deployment
    entrypoint: flows.py:my_flow
    work_pool:
      name: my-k8s-pool
      job_variables:
        image: <http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0>
        image_pull_policy: IfNotPresent
A couple caveats: * If your flow code is not baked into the image, Prefect still needs a way to access it at runtime * Prefer pinned tags like 
:1.0.0
instead of 
:latest
* If you update the same tag repeatedly, use 
image_pull_policy="Always"
Relevant docs: * `prefect_kubernetes.decorators` reference * Work pools * Kubernetes deployment docs If you want, I can also show you a full working example for: * ECR + Kubernetes work pool setup, or * flow code baked into the image vs pulled from Git/S3
n
@Marvin what if the uv project the flow belongs to is a subfolder of a bigger monorepo, which is what is pulled at runtime directly from github? How does the entrypoint/working directory work for Kubernetes decorator flows?
m
✅ thought for 134.6 seconds
Yes — this is exactly where entrypoint vs working directory starts to matter. Short version: * For code pulled from GitHub at runtime, the entrypoint is relative to the pulled repo contents * If your flow lives in a subfolder of a monorepo, the entrypoint should usually include that subfolder * The 
@kubernetes
decorator itself is mainly about *where the flow run executes*; it does not by itself redefine your repo layout or magically switch into your uv subproject * For monorepos, 
flow.from_source(...)
or deployment 
pull
steps are usually clearer than relying only on 
@kubernetes
Example monorepo:
Copy code
text
repo-root/
  services/
  libs/
  flows/
    pyproject.toml
    uv.lock
    my_flow.py
If GitHub pulls the whole repo, then your entrypoint is typically:
Copy code
text
flows/my_flow.py:my_flow
not just:
Copy code
text
my_flow.py:my_flow
because Prefect resolves it relative to the pulled source root unless you explicitly change the working directory first. Recommended pattern If the flow belongs to a subfolder uv project, use 
from_source(...).deploy(...)
and make the entrypoint explicit:
Copy code
python
from prefect import flow

@flow
def my_flow():
    print("hello")
    
if __name__ == "__main__":
    my_flow.from_source(
        source="<https://github.com/my-org/my-monorepo.git>",
        entrypoint="flows/my_flow.py:my_flow",
    ).deploy(
        name="my-flow",
        work_pool_name="my-k8s-pool",
        image="<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0>",
    )
A few important details: * If your Docker image already contains all dependencies, that’s great — but the flow source code still needs to line up with the runtime path Prefect uses * If the image also already contains the repo code at the correct path, you may not want runtime Git pulling at all * If Prefect does pull from GitHub at runtime, it clones the repo and resolves the entrypoint from that checkout About the working directory There are basically two cases: 1. Entrypoint includes the subfolder path * Example: 
flows/my_flow.py:my_flow
* This is the simplest and most reliable for monorepos 2. You change the working directory before resolving the entrypoint * Then you could use 
my_flow.py:my_flow
* This is more common with deployment 
pull
steps like 
set_working_directory
For example in `prefect.yaml`:
Copy code
yaml
pull:
  - prefect.deployments.steps.git_clone:
      repository: <https://github.com/my-org/my-monorepo.git>
      branch: main
  - prefect.deployments.steps.set_working_directory:
      directory: my-monorepo/flows

deployments:
  - name: my-flow
    entrypoint: my_flow.py:my_flow
    work_pool:
      name: my-k8s-pool
      job_variables:
        image: <http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0>
Without that 
set_working_directory
, you’d usually want:
Copy code
yaml
entrypoint: flows/my_flow.py:my_flow
Important uv / monorepo caveat If your 
uv
project is in a subfolder, Prefect does not automatically activate that subproject just because the flow file is there. So if your runtime behavior depends on: * 
pyproject.toml
in the subfolder * local package imports from that subfolder layout * 
uv run
semantics * relative file access then you should make sure one of these is true: 1. Your image already has the dependencies installed, and your flow imports work regardless of cwd 2. Your code is installed as a package in the image 3. You set the working directory to the uv project subfolder before execution In practice, the most robust setup is often: * bake dependencies into the ECR image * optionally bake the flow code in too * install the subproject as a package in the image * use explicit entrypoints That avoids depending on runtime cwd quirks. For 
@kubernetes
specifically The decorator is great for telling Prefect “run this flow on my Kubernetes work pool with these job variables”, like: ```python from prefect import flow
from prefect_kubernetes.decorators import kubernetes @kubernetes( work_pool="my-k8s-pool", image="123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0", ) @flow def my_flow(): ... ``` But for a monorepo + runtime Git source + uv subproject, I’d usually recommend a deployment/source-based approach over relying on the decorator alone, because it gives you much more explicit control over: * source location * entrypoint path * working directory * pull behavior Rule of thumb * If the repo root is cloned and your flow is in a subfolder: use 
subfolder/file.py:flow_fn
* If you explicitly change cwd to the subfolder first: use 
file.py:flow_fn
If you want, I can give you one of these next: 1. a complete 
prefect.yaml
example for monorepo + uv subfolder + existing ECR image 2. a Python-only 
from_source(...).deploy(...)
example 3. guidance on baking the monorepo subproject into the image so Git pull isn’t needed at runtime
n
@Marvin I don't care about the deployment, I'm talking about the Kubernetes decorator. what if i want to use it in a monorepo?
m
✅ thought for 8.5 seconds
Got it — for the 
@kubernetes
decorator itself in a monorepo: The decorator does not give you a way to set a repo subdirectory / working directory like a deployment pull step does. What it really does is submit the flow run to a Kubernetes work pool and pass job variables like 
image
, 
namespace
, etc. It’s about infrastructure, not source checkout semantics. So in a monorepo, the important distinction is: * 
@kubernetes(...)
controls where the flow runs * your Python execution/import context still depends on how that flow file is being invoked and what exists in the container at runtime If your flow file is inside a monorepo subfolder, then for 
@kubernetes
to work cleanly, one of these should be true: * the image already contains the monorepo code in a layout that preserves imports * the relevant subproject is installed into the image as a package * the runtime command/environment in the container already starts in a usable cwd Because with the decorator alone, there is not a built-in “run this flow from subdirectory `foo/bar`” parameter. Example monorepo:
Copy code
text
repo/
  libs/
  services/
  prefect_flows/
    pyproject.toml
    my_flow.py
If 
my_flow.py
does imports like:
Copy code
python
from prefect_flows.some_module import x
then your image needs to support that import path. The safest way is usually to install that subproject into the image rather than relying on cwd being 
repo/prefect_flows
. So instead of depending on working directory, prefer:
Copy code
dockerfile
COPY prefect_flows /app/prefect_flows
WORKDIR /app
RUN pip install -e /app/prefect_flows
or otherwise ensure the package is importable. Then your flow can be:
Copy code
python
from prefect import flow
from prefect_kubernetes.decorators import kubernetes

@kubernetes(
    work_pool="my-k8s-pool",
    image="<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:1.0.0>",
)
@flow
def my_flow():
    ...
Key point: with 
@kubernetes
, don’t design around “Prefect will cd into my monorepo subfolder”. That’s not really the model. Instead, design so that the container can run the flow without needing a special cwd, by doing one of: * install the subproject as a package * copy the code into a stable path in the image and make imports absolute * set 
PYTHONPATH
in the image if needed If you rely on relative file access like:
Copy code
python
open("./config.yaml")
that’s fragile in this setup. Better:
Copy code
python
from pathlib import Path

CONFIG = Path(__file__).parent / "config.yaml"
So the practical answer is: In a monorepo, the Kubernetes decorator can use your existing ECR image just fine, but it does not manage subdirectory entrypoint/cwd behavior for you. Make the monorepo subproject importable inside the image, and avoid depending on runtime cwd. If you want, I can sketch the recommended image layout for a monorepo subproject using 
uv
 so 
@kubernetes
works reliably.
n
@Marvin Just tell me what Prefect does under the hood to know how to run a flow with the Kubernetes decorator. How does it know where is the function to run?
m
✅ thought for 126.2 seconds
Under the hood, the 
@kubernetes
decorator does not primarily locate your flow by an entrypoint path like 
file.py:flow
. Instead, when you call the flow, Prefect takes the actual 
Flow
object in memory and submits that for remote execution. So this:
Copy code
python
@kubernetes(work_pool="my-pool")
@flow
def my_flow():
    ...
works roughly like this: 1. You call 
my_flow()
2. The decorator intercepts that call 3. Prefect creates a flow run 4. Prefect serializes the flow object itself using 
cloudpickle
5. It creates a bundle containing: * the serialized flow * flow run metadata * execution context * dependency metadata 6. That bundle is uploaded to storage used by the work pool 7. The Kubernetes worker starts a pod 8. The pod runs a Prefect command that downloads the bundle 9. Prefect deserializes the flow object 10. It executes the original flow function with the provided parameters So the answer to: “How does it know where the function is?” is: It usually doesn’t look it up by file path at runtime for the Kubernetes decorator path — it already has the function object and ships that object over as a serialized bundle. That’s the important distinction from deployment/source-based execution, where Prefect uses an entrypoint string and imports code from files or Git. With 
@kubernetes(...)
, the remote side runs something conceptually like:
Copy code
bash
uv run -m prefect.bundles.execute --key <bundle-key>
That bundle contains the serialized flow. So for decorator-based Kubernetes runs: * not primarily “import 
repo/subdir/file.py
, find `my_flow`” * instead “deserialize this flow object and run it” Why this matters for your monorepo question: * Prefect is not depending on repo-relative entrypoint resolution in the same way * But your container still needs an environment where deserializing and executing that flow works * That means the image still needs compatible Python/package dependencies * And if your flow references local modules/files in ways that require a specific filesystem layout, those still need to exist So although Prefect knows which function to run from the serialized flow object, that does not mean all referenced code/data magically comes along unless it’s part of what gets bundled or already exists in the image. In one sentence: For 
@kubernetes
, Prefect knows what to run because it serializes the in-memory flow object and sends that bundle to the Kubernetes job, rather than resolving the flow from an entrypoint string at runtime. If you want, I can next explain: * what parts of a monorepo flow are actually captured by that bundle, versus * what still must already exist in the image/filesystem
Open in Slack
PreviousNext
Powered by