<@ULVA73B9P> I am using a prefect Secret block to ...
# ask-marvin
j
@Marvin I am using a prefect Secret block to hold a JSON secret. The problem is, the block doesn't provide any feedback to the user as to what the JSON should look like. Walk me through the steps to create a new kind of prefect block called "SchemaSecret" which has a secret block inside it but also an unencrypted JSON schema that shows a prototype example. When the user creates a new block of type SchemaSecret, I want them to see the Schema as they're filling out a new value.
m
thought for 611.0 seconds
Great idea. In Prefect 3.x, the cleanest way to achieve this is to define a custom block that: - stores the actual JSON secret in an encrypted field (using
prefect.types.SecretDict
) - exposes a separate, unencrypted “schema/example” field (a plain dict) that the UI will render while the user enters the secret - uses field metadata so the schema appears first in the UI form Below are the step-by-step instructions and a complete, ready-to-register block. High-level plan - Don’t embed a
Secret
block inside your custom block. Instead, use
SecretDict
for the encrypted JSON field; it’s designed for this and the UI will redact values. - Add a plaintext
schema
field (dict) with a default/example. Prefect’s form renderer will show it right above the secret field. - Optionally add a simple validator to ensure the secret roughly matches the schema keys. 1) Create the block class Save this to a Python file (for example,
blocks/schema_secret.py
):
Copy code
from typing import Any, Dict
from pydantic import Field, model_validator
from prefect.blocks.core import Block
from prefect.types import SecretDict


class SchemaSecret(Block):
    """
    Store a JSON secret (encrypted) with a visible, unencrypted schema/example
    to guide users when creating/updating the block.
    """

    _block_type_name = "Schema Secret"
    _description = "An encrypted JSON secret alongside a plaintext schema/example."
    _code_example = '''
from prefect.types import SecretDict
from your_package.blocks.schema_secret import SchemaSecret

block = SchemaSecret(
    schema={
        "api_key": "sk-EXAMPLE",
        "endpoint": "<https://api.example.com>",
        "timeout": 30
    },
    secret=SecretDict({
        "api_key": "sk-live-****",
        "endpoint": "<https://api.example.com>",
        "timeout": 30
    })
)
block.save("my-schema-secret", overwrite=True)

loaded = SchemaSecret.load("my-schema-secret")
secret_values = loaded.secret.get_secret_value()
print(secret_values["api_key"])
'''

    # Plaintext field visible in the UI to guide users
    schema: Dict[str, Any] = Field(
        default_factory=lambda: {
            "api_key": "sk-EXAMPLE",
            "endpoint": "<https://api.example.com>",
            "timeout": 30
        },
        title="Schema / Example",
        description="A non-sensitive example/prototype of the expected JSON structure. Stored in plaintext.",
        json_schema_extra={"position": 0},
        examples=[{"api_key": "sk-EXAMPLE", "endpoint": "https://...", "timeout": 30}],
    )

    # Encrypted, redacted field for the actual secret JSON
    secret: SecretDict = Field(
        default_factory=lambda: SecretDict({}),
        title="Secret JSON",
        description="Paste your sensitive JSON matching the schema above. Stored encrypted and redacted in the UI.",
        json_schema_extra={"position": 1},
    )

    # Optional: basic validation that secret includes keys from the schema
    @model_validator(mode="after")
    def validate_secret_matches_schema(self):
        try:
            data = self.secret.get_secret_value()
        except Exception:
            # If we can't read the secret yet, skip strict validation
            return self

        if self.schema:
            missing = {k for k in self.schema.keys() if k not in data}
            if missing:
                raise ValueError(f"Secret is missing keys from schema: {sorted(missing)}")
        return self
Why this works -
secret: SecretDict
ensures the content is encrypted at rest and redacted in the UI/logs while still letting people paste JSON. -
schema: Dict[str, Any]
is plaintext and visible. The
json_schema_extra={"position": 0}
puts it above the secret field so the user sees it first. - The optional
model_validator
enforces basic key matching if you want stricter guardrails. 2) Register the block with Prefect Run the CLI to make the new block type available in your workspace:
Copy code
prefect block register -f blocks/schema_secret.py
Then check it’s registered:
Copy code
prefect block type ls
3) Create a SchemaSecret block in the UI - Go to Blocks in the Prefect UI - Click +, search for “Schema Secret” - You’ll see the Schema/Example field first (plaintext), then the Secret JSON field (redacted). Paste your real sensitive JSON into Secret JSON. 4) Use the block in code
Copy code
from prefect.types import SecretDict
from blocks.schema_secret import SchemaSecret

# Create or update
block = SchemaSecret(
    schema={"api_key": "sk-EXAMPLE", "endpoint": "<https://api.example.com>", "timeout": 30},
    secret=SecretDict({"api_key": "sk-live-abc123", "endpoint": "<https://api.example.com>", "timeout": 60})
)
block.save("my-schema-secret", overwrite=True)

# Load and use
cfg = SchemaSecret.load("my-schema-secret")
secret_values = cfg.secret.get_secret_value()
print(secret_values["api_key"])
Optional: package for auto-discovery If you’re distributing this as a package, you can auto-register blocks by adding an entry point in your
pyproject.toml
so Prefect discovers your collection:
Copy code
[project.entry-points."prefect.collections"]
your-collection = "your_package"
Helpful docs - Custom blocks: Custom blocks - Blocks overview: Blocks concepts - Storing secrets: Store secrets - Form-building tips: Form building - Block API reference: prefect.blocks.core and prefect.blocks.system Would you like the validator to be stricter (e.g., type-check values against the example), or is the visible schema + basic key check sufficient?