<@ULVA73B9P> Hi, if I have 2 k8s namespaces "prod"...
# marvin-ai
a
@Marvin Hi, if I have 2 k8s namespaces "prod" and "staging", do I need 2 workers for running prefect flows as k8s jobs?
m
thought for 119.6 seconds
No — not necessarily. In Prefect 3.x, you usually need: at least one worker per work pool, not one worker per Kubernetes namespace. So whether you need 1 or 2 workers depends on how you set things up: * 1 worker is enough if it can submit jobs into both
prod
and
staging
* 2 workers are only needed if you want strict separation, like one work pool/worker per environment A common setup is: *
prod
deployment -> work pool/job variables use namespace
prod
*
staging
deployment -> work pool/job variables use namespace
staging
The namespace is part of the Kubernetes job configuration, so it can be set per deployment or per work pool. Example using deployment job variables:
Copy code
yaml
deployments:
  - name: my-flow-prod
    entrypoint: flows.py:my_flow
    work_pool:
      name: k8s-pool
      job_variables:
        namespace: prod

  - name: my-flow-staging
    entrypoint: flows.py:my_flow
    work_pool:
      name: k8s-pool
      job_variables:
        namespace: staging
Or you can make two work pools (
k8s-prod
,
k8s-staging
) with fixed namespaces and run separate workers for cleaner isolation. A few important caveats: * The worker/service account must have RBAC permissions to create jobs in both namespaces if you use one worker * If you want stronger isolation, separate workers/work pools are often easier to reason about * A worker polls a single work pool via
prefect worker start --pool <pool-name>
so if you use two work pools, you generally run one worker for each pool Verified CLI:
prefect worker start --pool <pool-name>
Helpful docs: * Workers * Work pools * Prefect Kubernetes integration If you want, I can show you the recommended setup for either: * single worker, two namespaces, or * two workers, one per environment
a
@Marvin Does the prefect helm chart allow setting up RBAC for multiple namespaces or should they be manually created?
m
thought for 103.5 seconds
The Helm chart helps partially, but for multiple namespaces you should expect to create some RBAC manually. Short version: * The
prefect-worker
Helm chart creates a
ServiceAccount
,
Role
, and
RoleBinding
* That built-in RBAC is basically for one namespace * If your worker needs to submit/watch jobs in multiple namespaces, you usually need to create additional `Role`/`RoleBinding` objects in each target namespace yourself * If you want true cluster-wide access, you'd typically create a
ClusterRole
and
ClusterRoleBinding
manually — the chart does not create those for you A useful distinction: * Where jobs run is controlled by the deployment/work pool job configuration (
namespace
) * What the worker is allowed to access is controlled by Kubernetes RBAC So for
prod
+
staging
, a common setup is: 1. Install the worker in one namespace, e.g.
prefect
2. Let the chart create its default service account/RBAC there 3. Manually add
Role
+
RoleBinding
in
prod
4. Manually add
Role
+
RoleBinding
in
staging
5. Point the bindings at the worker's service account You may also want the worker observer configured to watch multiple namespaces. The chart supports config for that, but not automatic RBAC creation for all those namespaces. Relevant references: * prefect-worker Helm chart * prefect-worker values.yaml * Prefect Kubernetes guide If helpful, I can give you a ready-to-apply example for: * one worker service account with access to both
prod
and
staging
, or * a cluster-wide ClusterRole/ClusterRoleBinding approach