<@ULVA73B9P> When we use endpoint - /api/work_pool...
# ask-community
b
@Marvin When we use endpoint - /api/work_pools/filter - it also retuns the env variables, Is there a way to not retur that because that conatains my secrets
m
thought for 335.6 seconds
Short answer: there isn’t a server-side switch today to exclude or redact the
env
from
/api/work_pools/filter
. That endpoint returns the full work pool model, including the
base_job_template
and its
job_configuration.env
if you’ve put values there. If sensitive values are stored in the pool template, they will be returned by the API. Recommended approaches to avoid exposing secrets: - Don’t store secrets in the work pool’s
base_job_template.env
- Instead, inject secrets at runtime via your infrastructure: - Kubernetes: reference Kubernetes Secrets (e.g., envFrom/secretKeyRef) so only the secret names appear in the template, not values. - Docker/ECS/etc.: use the platform’s secret mechanism or pass env vars to the runtime container from your orchestrator; avoid putting the actual values in Prefect’s pool config. - Use Prefect Secret blocks in your flow code - Store the secret in a Prefect Secret block (encrypted at rest) and load it in your flow; don’t propagate it through the pool template. - Example:
Copy code
from prefect.blocks.system import Secret

    # One-time setup
    Secret(value="super-secret").save("my-api-token", overwrite=True)

    # In your flow
    @flow
    def run():
        token = Secret.load("my-api-token").get()
        ...
- Docs: Store secrets - Use templating to reference env at render time (keeps values out of the database) - In your pool’s
base_job_template
, set env values to placeholders that read from the worker’s environment at runtime, e.g.:
Copy code
{
  "job_configuration": {
    "env": {
      "MY_SECRET": "{{ env.MY_SECRET }}"
    }
  }
}
- This way, the API returns the placeholder string, not the secret value. Ensure the worker’s environment actually has
MY_SECRET
set. - Worker/template docs: Manage work pools and Custom worker A few additional notes: - Prefect Variables are not intended for secrets (no sensitive/redaction flag). Avoid putting secrets there: Variables - Work pools API reference (to confirm the returned fields): Work pools API If you can share which worker type you’re using (Kubernetes, Docker, etc.) and whether this is Prefect Cloud or a self-hosted server, I can suggest a concrete pattern and example template for that runtime.
b
@Marvin I am just creating deployments for my normal python scripts so that I can schedule them. I am creating deployments - can I inject secrets at that time while creating deployments in any way ? I am not wanting to change my existing scripts
m
thought for 452.8 seconds
Short answer: Yes — without changing your scripts, the safest way is to inject secrets as environment variables on the deployment (or the work pool/worker) so your script can read them as it does today. Here are your options in Prefect 3.x: - Set env vars at deployment creation (CLI) - You can override job variables at deploy time, including env vars:
Copy code
prefect deploy ./path/to/script.py:flow_func \
      --name my-deployment \
      --job-variable env.MY_SECRET=super-secret-value
- Note: this stores the value with the deployment in Prefect’s API and may be visible to users with access. Also consider your shell history. Use only if that’s acceptable. - Docs: Customize job variables - Define env vars in prefect.yaml (checked into your repo) - Add them under job_variables.env for the deployment:
Copy code
deployments:
      - name: my-deployment
        entrypoint: path/to/script.py:flow_func
        work_pool:
          name: process-pool
        job_variables:
          env:
            MY_SECRET: "super-secret-value"
- Docs: prefect.yaml reference - Set env vars at the work pool or worker level (recommended for sensitive values) - Process work pool: set env vars on the machine/service that runs
prefect worker start
— flow runs inherit them. - Docker/Kubernetes work pools: add env to the base job template so every run gets them (or use your orchestrator’s secret store, e.g., Kubernetes Secrets). - Docs: Customize job variables (work pool base job template) If you were open to a tiny code change later, Prefect provides Variables and a Secret block you can load at runtime (keeps secrets out of deployments), but that does require reading them in code: - Variables/Secret concepts: Variables concept Questions to tailor the best option: - How does your script currently get secrets (env vars, config file, CLI args)? - Which work pool/worker type are you using (process, docker, kubernetes)? - Any constraints about storing secrets in Prefect vs only on the worker/infrastructure?