<@ULVA73B9P>, should we use <https://github.com/Pr...
# ask-marvin
t
@Marvin, should we use https://github.com/PrefectHQ/prefect-operator/blob/main/PrefectDeployment.md to deploy our deployments in kubernetes or use prefect.yaml via prefect deploy
m
thought for 121.1 seconds
Short answer: Use
prefect.yaml
with
prefect deploy
and run your flows on a Kubernetes work pool via the Kubernetes worker. Do not use the PrefectDeployment CRD from prefect-operator for Prefect 3.x. Why: - In Prefect 3.x, the supported and recommended model is workers + work pools. A Kubernetes worker running in your cluster pulls work from a Kubernetes work pool and launches each flow run as a Kubernetes Job. - The prefect-operator/PrefectDeployment CRD is not a first-class, supported path for 3.x and isn’t recommended for production. - The
prefect.yaml
+
prefect deploy
approach is version-controllable, CI/CD friendly, and fully supported. Minimal setup (Prefect 3.x): 1) Create a Kubernetes work pool
Copy code
prefect work-pool create --type kubernetes my-k8s-pool
2) Run the Kubernetes worker in your cluster (recommended via Helm)
Copy code
helm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm install prefect-worker prefect/prefect-worker \
  --namespace prefect \
  -f values.yaml
3) Define deployments in
prefect.yaml
and deploy
Copy code
prefect deploy --help
prefect deploy --all
# or
prefect deploy path/to/flow.py:my_flow -n my-deployment -p my-k8s-pool
Good references: - Deployments (3.x): Deployments - Kubernetes worker + work pools: Kubernetes work pool guide - Workers overview: Workers - prefect.yaml: prefect.yaml - Helm chart: Prefect Helm repo If you’re on a specific K8s distro (EKS/GKE/AKS) or need custom Job templates (node selectors, volumes, SA, etc.), I can share a minimal
values.yaml
and work pool job template example. Are you using Prefect Cloud or self-hosted, and do you already have a container image for your flows?
t
@Marvin, why is it not supported in 3.x
m
thought for 118.7 seconds
Great question. Short version: Prefect 3.x’s official, supported architecture is workers + work pools, not a Kubernetes Operator/CRD. The prefect-operator exists, but it’s not part of the documented or supported 3.x path. Why it isn’t supported as the primary 3.x path: - Architecture choice: 3.x standardized on a worker/work-pool model that works across Kubernetes, Docker, ECS, Cloud Run, etc. A K8s-only Operator adds a parallel, K8s-specific control plane that Prefect would need to maintain alongside workers. - Simplicity and decoupling: Workers are stateless, easy to run (e.g., via Helm), and keep Prefect’s control plane separate from your cluster. An Operator tightly couples orchestration to Kubernetes via CRDs. - Consistency and supportability: One consistent model (workers/work pools +
prefect deploy
) makes docs, support, and features uniform across infrastructures. The operator is a separate repo and is not referenced in the official 3.x docs. - Maturity and documentation: The 3.x docs focus on workers/work pools; the operator isn’t documented as an official pathway. That’s a signal that it’s not at the same support/maturity level for production guidance. What’s officially recommended for Kubernetes in 3.x: - Use a Kubernetes work pool + Kubernetes worker and deploy flows via `prefect.yaml`/`prefect deploy`. - Kubernetes guide: Kubernetes work pool guide - Workers concept: Workers - Deployments: Deployments - prefect.yaml: prefect.yaml - Helm chart for the worker: Prefect Helm repo About the operator: - Repo: prefect-operator and its PrefectDeployment.md - It can work for Kubernetes-native, CRD-driven shops, but it’s outside the officially documented 3.x path and not what Prefect Support will point you to for production. If you’ve already invested in the operator, it may be fine for specific needs—but for a supported, well-documented, cross-infrastructure path, stick with workers/work pools +
prefect deploy
. If helpful, I can share a minimal
values.yaml
and a Kubernetes work pool job template that covers image, service account, volumes, and resource requests.
t
@Marvin give me sources on where the 3.x doesn't support the CRD
m
thought for 177.0 seconds
Totally fair ask. There isn’t a single line that says “Prefect 3.x does not support a CRD,” but the official 3.x docs and release notes only document and recommend workers + work pools (no CRDs/operator). Here are authoritative sources you can point to: - Prefect 3.x Kubernetes guide (only documents workers/work pools; recommends Helm for the worker) - Kubernetes work pool guide - Quote: “After you create a cluster and work pool, the next step is to deploy a worker… The recommended method for deploying a worker is with the Prefect Helm Chart.” (and links to Prefect Helm chart) - Workers are the standard in 3.x (no mention of a Kubernetes Operator/CRDs) - What’s new in Prefect 3.0 - Quote: “Workers were first introduced in Prefect 2.0 as next-generation agents, and are now standard in Prefect 3.0.” - Migration guidance focuses on workers, not an operator - Upgrade agents to workers - Quote: “If you host your agents in a Kubernetes cluster, you can use the Prefect worker Helm chart to host workers in your cluster.” (no CRD/operator path documented) - Core concepts emphasize work pools/workers, not CRDs - Workers - Work pools - These pages define the supported, recommended execution model; there’s no CRD/operator alternative presented. - Release notes for 3.x contain updates for Kubernetes worker and work pools; no CRD/operator references - Prefect 3.x releases - You can search through 3.x notes—there are entries for Kubernetes worker RBAC/docs, but no mentions of a PrefectDeployment CRD or operator in 3.x. - The prefect-operator is a separate repo and not referenced by the 3.x docs - prefect-operator repo - Its CRD docs (e.g., PrefectDeployment.md) live in that repo, not in the official 3.x docs. The absence from the 3.x docs is the key signal it’s not the documented/supported path for 3.x deployments. Summary: The official 3.x documentation and release notes describe and recommend workers + work pools (with Helm for Kubernetes) and make no mention of a CRD/operator path. The operator exists in a separate repo, but it’s not part of the documented or supported 3.x deployment model.
t
@Marvin if i use crd, i can't use worker and work pools?
m
thought for 76.6 seconds
Yes—you still use workers and work pools with the CRD approach. How it fits together with the operator (Prefect 3.x): - PrefectWorkPool CRD - Creates/updates a Prefect work pool via the Prefect API (includes the base job template) - Creates a Kubernetes Deployment that runs Prefect worker pods (e.g.,
prefect worker start --pool <name> --type <type>
) - PrefectDeployment CRD - Creates/updates a Prefect Deployment in the Prefect API - References a work pool (created via PrefectWorkPool CRD or otherwise) - Does not launch Kubernetes Jobs itself; the worker does that So: - You can use the CRDs, but they still rely on the standard 3.x model where workers poll work pools and create flow-run Jobs. - Without worker pods running for the referenced work pool, flow runs won’t execute. If you prefer a pure Prefect path without CRDs, you can: - Create a Kubernetes work pool in Prefect - Run the Kubernetes worker via the official Helm chart - Deploy with
prefect.yaml
+
prefect deploy
If you’d like, I can share a minimal pair of CRDs (PrefectWorkPool + PrefectDeployment) that mirror the same outcome as the Helm +
prefect deploy
path.
t
@Marvin what's the overall cons for the PrefectDeployment CRD? is it that it's not paired with actual prefect? so the versions might be mismatched?
m
thought for 23.4 seconds
Great question. The PrefectDeployment CRD can work, but here are the main cons to be aware of in Prefect 3.x: - Not the officially documented path for 3.x - Official docs recommend workers + work pools (with Helm for Kubernetes workers) and make no mention of CRDs/operators. - Links: - Kubernetes work pool guide - Workers - Work pools - prefect.yaml - Version and compatibility management is on you - You’ll need to keep three layers compatible: - Prefect API (Cloud or Server) - Operator version (its CRD schema and controller logic) - Worker and flow runtime images (Prefect Python versions) - New Prefect features can land in the API/SDK/worker first; the operator may lag in exposing them. Pin versions and test upgrades. - Operator repo: prefect-operator (e.g., PrefectDeployment.md) - Dual source of truth and drift risk - If you also use
prefect deploy
, the UI, or Terraform to manage the same deployments/work pools, you can create configuration drift. With CRDs, Kubernetes becomes your desired state; with
prefect deploy
, Prefect is the source of truth. Pick one for each resource to avoid conflicts. - Operational complexity - You’re operating an extra controller (RBAC, upgrades, failure modes, reconciliation loops). Debugging involves operator logs, Prefect API logs, worker logs, and Kubernetes Job logs. - The official Helm chart for workers is simpler if your only goal is to run flows in K8s: Prefect Worker Helm chart. - Feature exposure may lag behind `prefect.yaml`/CLI - The
prefect.yaml
+
prefect deploy
path has first-class support for build/push/pull steps, templating, and multi-deployment workflows. - The CRD must map these capabilities into its schema; new features or parameters may not appear immediately. - Scope and limitations - The operator interops with workers/work pools (it does not replace them). It creates/updates work pools via the Prefect API and deploys worker pods; workers still launch Jobs. - Push/managed work pool types (no worker needed) aren’t the operator’s target; it focuses on hybrid patterns that use workers. - Community support and docs footprint - Fewer end-to-end, production-grade examples and troubleshooting docs compared to the worker/work-pool +
prefect deploy
approach which is the one documented by Prefect. To your specific question: yes, version mismatch is a real consideration. You should: - Pin a specific operator release and match it with your Prefect Cloud/Server API version expectations. - Pin the worker image version (Prefect version) and your flow runtime images; test upgrades in staging. - Avoid mixing CRD-managed and CLI/UI-managed changes to the same deployment/work pool. If you value Kubernetes-native GitOps and are comfortable managing the operator lifecycle, the CRD can be useful. If you want the most supported, documented, and straightforward path, use Kubernetes workers + work pools with
prefect.yaml
and
prefect deploy
.
t
@Marvin can we use CRD overall and not Prefect Deploy if we're comfortable with kubernetes native
m
thought for 16.4 seconds
Yes—with caveats. You can manage everything via the CRDs and skip
prefect deploy
if you’re comfortable with a Kubernetes‑native, GitOps flow. What that looks like: - Use PrefectWorkPool CRD to: - Create/update a Prefect work pool (type typically
kubernetes
) - Deploy/scale the corresponding Kubernetes worker pods (these still do the execution) - Use PrefectDeployment CRD to: - Create/update the Prefect Deployment (entrypoint, parameters, schedules) and point it at your work pool - Provide Prefect API URL/key via K8s Secrets/CRD fields per the operator docs (works with Prefect Cloud or self-hosted) Important caveats/tradeoffs: - Single source of truth: If you go CRD-first, avoid changing the same deployments/work pools via
prefect deploy
, UI, or Terraform to prevent drift. - Image/builds are on you: The CRDs won’t build/push images. You’ll handle that in CI and reference the image in your job template or deployment config. - Feature parity/lag: New Prefect features typically land in the SDK/CLI/docs first; the operator may trail in exposing every knob. - Version pinning: You’re now managing compatibility between - Prefect Cloud/Server (API) - Operator/CRD versions - Worker and flow runtime images (Prefect Python versions) Pin and test upgrades. Authoritative references: - Prefect Operator repo: prefect-operator - PrefectDeployment CRD doc: PrefectDeployment.md - Workers/work pools concepts (still apply): Workers, Work pools - Official (non-CRD) K8s guide for comparison: Kubernetes work pool guide If you share whether you’re on Prefect Cloud or self-hosted, I can point you to the specific operator examples for API credentials and a minimal WorkPool + Deployment CRD pair.
t
@Nate was wondering on your thoughts on using PrefectDeploymentCRD vs prefect deploy
@Marvin im using prefect self-hosted
m
thought for 116.4 seconds
You can use the CRD path with self-hosted Prefect, yes. There are two common patterns: - Operator-managed server: Run Prefect Server in your cluster via the operator’s PrefectServer CR, then create a PrefectWorkPool CR and PrefectDeployment CR that reference it. - External server: Point the operator at your existing self-hosted Prefect Server (API URL/key), then create PrefectWorkPool and PrefectDeployment CRs that reference that external server. A few important notes before I paste any YAML: - The operator is a separate repo from core Prefect. Its CRD fields can change by version. To avoid giving you incorrect specs, I need your operator version (or a link to the chart/manifests you’re using). - The operator approach still uses workers/work pools under the hood; the PrefectWorkPool CR both registers the pool with your server and deploys worker pods that poll that pool. - You’ll need to provide API connectivity/secrets for the worker pods to reach your self-hosted API (URL and, if applicable, auth). Authoritative references (these have the current CRD schemas and examples): - Repo: prefect-operator - PrefectDeployment CRD doc: PrefectDeployment.md - (Typically) examples/samples dir in the repo for PrefectServer, PrefectWorkPool, and end-to-end manifests If you share: - Operator version you’re targeting - Whether your Prefect Server is already running (and its API URL) or you want the operator to run it - Namespace and desired work pool type (likely kubernetes) - Worker image/tag you want to run …I’ll draft minimal, ready-to-apply CRDs for PrefectServer (if needed), PrefectWorkPool, and PrefectDeployment that match your setup. If you prefer the officially documented self-hosted path without the operator, here are the Helm charts: - Prefect Server (self-hosted): prefect-server Helm chart - Kubernetes Worker: prefect-worker Helm chart
t
@Marvin is the prefectdeployment crd maintained long term
m
thought for 19.0 seconds
Short answer: There’s no published, long‑term support guarantee for the PrefectDeployment CRD. The operator is a separate repo that’s actively maintained, but it isn’t part of the officially documented 3.x path, so you should treat it as optional/advanced rather than GA with a formal LTS promise. What we can say with sources: - Official 3.x docs recommend workers + work pools (no operator/CRD mentioned): - Kubernetes guide: Kubernetes work pool guide - Workers: Workers - Deployments: Deployments - The operator exists and is maintained in a separate repo, with its own releases: - Repo: prefect-operator - Releases: prefect-operator releases - CRD doc: PrefectDeployment.md Implications: - Because it’s not in the official 3.x docs and not referenced in Prefect release notes, there’s no explicit long‑term support/GA commitment to the PrefectDeployment CRD. It may lag new features or change independently of core Prefect. - If you adopt it, plan for: - Pinning operator + Prefect versions, and testing upgrades - Managing compatibility across Prefect Server, operator, worker images, and your flow images - A Kubernetes‑native management plane (CRDs) that can drift from Prefect‑managed state if you also use `prefect deploy`/UI/Terraform If you need the most stable, officially supported path, use
prefect.yaml
+
prefect deploy
with a Kubernetes work pool and worker. If you prefer the CRD for GitOps reasons, it can work—just adopt it with the above cautions and version pinning. If you’d like, I can help you evaluate risk based on your upgrade cadence and support requirements, or draft a migration plan that keeps you close to workers/work pools so you can pivot if needed.