https://prefect.io logo
Join Slack
Powered by
Hi <@ULVA73B9P>! I’m deploying self-deploy prefect...
# ask-marvin
e
Hi @Marvin! I’m deploying self-deploy prefect (with kubernetes), with infra-as-code as much as possible. I want to register system blocks (json, secret, ...). What are the alternatives to :
Copy code
prefect block register -m prefect.blocks.system
I can use terraform, or pulumi.
m
✅ thought for 451.4 seconds
n
variables (can be any JSON value, including a string) have superseded JSON / string blocks but there are tf resources for both variables and blocks (e.g. Secret block)
e
Hey @Nate ! Thanks for these resources, I don’t need to add a block but to register a block type.
n
there's a separate resource for block types
e
Hmmm... I’ll try sth like :
Copy code
resource "prefect_block_type" "prefect-block-type-json" {
  name = "json"
  slug = "json"
}
n
variables (can be any JSON value, including a string) have superseded JSON / string blocks
so, this is why i mentioned this
m
Great question — and nice job aiming for IaC. Assuming Prefect 3.x: the JSON block from 2.x is gone. Use Variables for JSON/structured config, and system blocks now include Secret, Webhook, and LocalFileSystem. Alternatives to `prefect block register -m prefect.blocks.system`: 1) Server API (best for IaC/bootstrapping) - POST the protected system blocks install endpoint. This installs Secret, Webhook, and LocalFileSystem as protected types.
Copy code
curl -X POST "$PREFECT_API_URL/api/block_types/install_system_block_types" \
  -H "Authorization: Bearer $PREFECT_API_KEY"
- Code references: - API route: POST /block_types/install_system_block_types - Implementation: installs Webhook, Secret, LocalFileSystem 2) CLI in a Kubernetes init container/Job - Still valid in 3.x:
Copy code
prefect block register -m prefect.blocks.system      # Secret
prefect block register -m prefect.blocks.webhook     # Webhook
prefect block register -m prefect.filesystems        # LocalFileSystem
- Verified CLI help:
Copy code
prefect block register --help
3) Python API (init script or job) - You can programmatically register types:
Copy code
import asyncio
from prefect.blocks.system import Secret
from prefect.blocks.webhook import Webhook
from prefect.filesystems import LocalFileSystem

async def main():
    for cls in (Secret, Webhook, LocalFileSystem):
        await cls.register_type_and_schema()
asyncio.run(main())
- Note: saving an instance will auto-register the type/schema:
Copy code
from prefect.blocks.system import Secret
Secret(value="my-secret").save("prod-secret", overwrite=True)
Code reference for auto-register in save(): Block.save registers type+schema automatically Terraform / Pulumi patterns - Terraform (simple, idempotent-enough for bootstrapping)
Copy code
resource "null_resource" "install_system_blocks" {
  provisioner "local-exec" {
    command = "curl -sS -X POST ${var.prefect_api_url}/api/block_types/install_system_block_types -H 'Authorization: Bearer ${var.prefect_api_key}'"
  }
}
- Then create block documents (e.g., a Secret) either: - via a small Python script run in a provisioner - or using an HTTP resource to POST to /api/block_documents (you’ll need to fetch the Secret block_type_id and block_schema_id first) - Pulumi - Use a Command/HTTP call in your stack to POST the same endpoint before any block document creation. - Then create block documents in code, e.g.:
Copy code
# run as a step in your Pulumi program or as a pre-step script
import asyncio
from prefect.client.orchestration import get_client
from prefect.blocks.system import Secret

async def setup():
    async with get_client() as client:
        # ensure protected system blocks exist (server-side)
        await client.request("POST", "/block_types/install_system_block_types")
        # create a secret block document
        await Secret(value="super-secret").save("db-password", overwrite=True)

asyncio.run(setup())
Using Variables instead of the old JSON block - Variables are the 3.x replacement for JSON config:
Copy code
prefect variable set my-config '{"key": "value"}'
- Load in code:
Copy code
from prefect.variables import get_variable
cfg = get_variable("my-config")
- CLI verified:
Copy code
prefect variable --help
Source links for the system blocks - Secret: prefect.blocks.system - Webhook: prefect.blocks.webhook - LocalFileSystem: prefect.filesystems Quick guidance
- For self-hosted Kubernetes with IaC, I recommend calling POST /block_types/install_system_block_types once at cluster/bootstrap time, then programmatically create any block documents you need (or rely on .save() auto-registration when you create them). - For JSON-like config, prefer Variables. If you share whether you’re on Prefect Cloud vs. self-hosted Orion API (and how you bootstrap the cluster), I can give a tailored Terraform/Pulumi snippet that fits your pipeline exactly.
n
we purposefully removed the JSON block type. variables have superseded the JSON block type, since variables can do the same thing without a separate type registration
e
and what about secrets ? variable can not behave like secret (not shown in the UI) right ?
n
but there are tf resources for both variables and blocks (e.g. Secret block)
variables are for things you'd be comfortable showing plaintext secrets are encrypted server-side and obfuscated in the UI, so we still have a block type for secrets
e
so I still need to register the block type secret, right ?
n
i wouldn't think so, unless you've deleted the default ones block type registration is generally only necessary when you've written your own custom 
Block
subclass that is not already defined in prefect
e
I’m pretty sure that when I deploy a self-hosted prefect, by default there is absolutely no block type available.
n
Copy code
» docker run -p 4200:4200 --rm -d prefecthq/prefect:3.6.2-python3.14 -- prefect server start --host 0.0.0.0
15ef8a0f5b9b560277e2c1da742a436ce229b2f91ef8f6fab0b3dc29c128a040

» uvx prefect config view
🚀 you are connected to:
<http://127.0.0.1:4200>
PREFECT_PROFILE='oss'
PREFECT_API_URL='<http://127.0.0.1:4200/api>' (from profile)
PREFECT_LOGGING_LEVEL='INFO' (from profile)

» uvx prefect block type ls | rg secret
│ aws-secret                           │ Manages a secret in     │ prefect block create     │
│                                      │ AWS's Secrets Manager   │ aws-secret               │
│ gcpsecret                            │ Manages a secret in     │ prefect block create     │
│                                      │ Google Cloud Platform's │ gcpsecret                │
│ secret                               │ A block that represents │ prefect block create     │
│                                      │ a secret value          │ secret
do you have an example showing that this is the case?
e
Ok I just reinstalled prefect in local following this page, I see many block types. I need to investigate now, why when I install it with pulumi on my remote kube cluster I don’t see any block types. Thanks @Nate !
Wow, after a redeploy with pulumi every block type appear coorectly 😲 I must have used a wrong helm version last time. Problem solved. Thanks again @Nate!
n
catjam
Open in Slack
PreviousNext
Powered by