Kerem Kaynak
12/18/2025, 7:24 PMMarvin
12/18/2025, 7:24 PMMarvin
12/18/2025, 7:27 PMprefect server start
- Prefect will use a local SQLite DB by default and expose the API at http://localhost:4200/api and the UI at http://localhost:3000.
- Ensure your shell points to the local API when running code in another terminal:
export PREFECT_API_URL=<http://localhost:4200/api>
2) Create and save a block
- You can use built-in blocks or define your own. Example using the built-in Secret:
from prefect.blocks.system import Secret
# create and save a block document
Secret(value="super-secret-token").save("my-api-token", overwrite=True)
- Custom block example:
from prefect.blocks.core import Block
from pydantic import SecretStr
class MyCredentials(Block):
api_key: SecretStr
base_url: str = "<https://api.example.com>"
# save a named instance
MyCredentials(api_key="abc123", base_url="<https://api.example.com>").save("dev-creds", overwrite=True)
3) Load a block in your flows
- In a sync flow:
from prefect import flow
from prefect.blocks.system import Secret
@flow
def use_secret():
token = Secret.load("my-api-token").get()
print(f"Loaded token of length: {len(token)}")
use_secret()
- In an async flow, use `aload`:
from prefect import flow
from prefect.blocks.system import Secret
@flow
async def use_secret_async():
secret = await Secret.aload("my-api-token")
print(secret.get())
use_secret_async()
4) View and manage blocks
- In the UI: open http://localhost:3000, go to Blocks, then Catalog (create) or Documents (list existing).
- From Python: list/inspect via the client if needed.
- From CLI: use the prefect block command group (run prefect block --help to see what’s available on your version).
Where blocks are stored locally
- If you’re running prefect server start, blocks are stored in the server’s database (SQLite by default, typically under ~/.prefect/prefect.db).
- Without a running server (or without PREFECT_API_URL pointing at one), Prefect may use an ephemeral in-memory API — any saved blocks will be lost when the process ends.
Export/import during local dev
- You can serialize block instances (careful with secrets) to move between environments:
from prefect.blocks.system import Secret
# export
s = Secret.load("my-api-token")
json_str = s.model_dump_json() # does not include raw secret by default
# import
restored = Secret.model_validate_json(json_str)
restored.save("my-api-token-copy", overwrite=True)
Gotchas and tips
- In async flows, prefer await MyBlock.aload("name").
- The Python class must be importable where you load it (the server stores the data, not the class code).
- If you see “Block document not found”, verify your code’s PREFECT_API_URL matches the server you saved to.
Helpful links
- Prefect UI (local): Open local UI
- Prefect API (local): Local API endpoint
- Blocks in the UI: Blocks UI
Want me to tailor this to your setup? Are you using the local server only, or Prefect Cloud too?Kerem Kaynak
12/18/2025, 7:31 PMMarvin
12/18/2025, 7:31 PMMarvin
12/18/2025, 7:32 PMprefect cloud login, not in code.
Recommended patterns
1) Create secrets as blocks in Prefect Cloud (no code)
- In the Cloud UI (Blocks > Catalog), create a Secret (or the relevant credential block) and give it a name like my-api-token.
- This keeps secrets out of code and your repo; Prefect encrypts secret fields at rest and scrubs them in logs.
2) Load the block in your flow
- Sync flow:
from prefect import flow
from prefect.blocks.system import Secret
@flow
def run():
token = Secret.load("my-api-token").get()
# use token (avoid logging or printing it)
- Async flow:
from prefect import flow
from prefect.blocks.system import Secret
@flow
async def run_async():
secret = await Secret.aload("my-api-token")
token = secret.get()
3) Authenticate to Prefect Cloud without embedding credentials in code
- Use prefect cloud login (stores auth locally) OR set env vars:
export PREFECT_API_URL=<https://api.prefect.cloud>
export PREFECT_API_KEY=your_cloud_api_key
- In CI/agents, use a service account API key stored in your CI secret store or worker environment, not in code.
4) Local development options without secrets in code
- Point your local dev to Prefect Cloud so your code loads the same Cloud blocks:
export PREFECT_API_URL=<https://api.prefect.cloud>
export PREFECT_API_KEY=your_cloud_api_key
- Or, if you prefer a local server for dev, bootstrap local blocks from environment variables (still no secrets in code):
import os
from prefect.blocks.system import Secret
Secret(value=os.environ["MY_API_TOKEN"]).save("my-api-token", overwrite=True)
Then set MY_API_TOKEN only in your local shell or .env (not committed), and load the block by name in your flows.
5) Rotation and environment separation
- Rotate by updating the block document in Cloud; your flows pick up the new value without code changes.
- Keep separate blocks per environment (e.g., my-api-token-dev, my-api-token-prod) and/or separate workspaces in Cloud. Reference the appropriate block name from each deployment/flow.
6) Optional: Use a cloud secret manager via a Prefect block
- If your org standard is AWS Secrets Manager, GCP Secret Manager, or Azure Key Vault, use the corresponding Prefect integration block to fetch secrets at runtime. Your code still references a block name; the block reads the secret from your provider.
Common pitfalls to avoid
- Printing or logging secret values (even length can sometimes leak info).
- Saving blocks programmatically with raw literals in code. If scripting block creation, read secrets from environment variables.
- Mixing local server and Cloud without adjusting PREFECT_API_URL — you’ll get “Block document not found” if pointed at the wrong API.
If you’d like, tell me which services you need creds for (e.g., AWS, GCP, DB) and your preferred workflow (Cloud-only vs. local dev + Cloud), and I can provide a tailored block setup and a bootstrap script that avoids secrets in code.Kerem Kaynak
12/18/2025, 7:37 PMMarvin
12/18/2025, 7:37 PMMarvin
12/18/2025, 7:38 PM# set in your local shell only (or a .env that is not committed)
export LOCAL_API_TOKEN=super-secret-for-local
# run once to create/update the local block
from prefect.blocks.system import Secret
import os
Secret(value=os.environ["LOCAL_API_TOKEN"]).save("my-api-token", overwrite=True)
- Your flows can then load my-api-token locally:
from prefect.blocks.system import Secret
from prefect import flow
@flow
def run_local():
token = Secret.load("my-api-token").get()
...
2) Cloud runs (no Cloud Secrets)
- Point your runtime to Prefect Cloud via env:
export PREFECT_API_URL=<https://api.prefect.cloud>
export PREFECT_API_KEY=... # service account key in CI/worker env
- Provide runtime secrets via your preferred method:
- Environment variables on the worker/CI (e.g., MY_API_TOKEN)
- Your cloud secret manager (AWS/GCP/Azure) using your standard SDK or a Prefect integration block that reads from those managers (the secret itself stays in your provider, not Prefect Cloud)
3) One small helper to unify both paths
- Use a single accessor so your flow code never hardcodes secrets or cares where they came from:
import os
from prefect.blocks.system import Secret
def get_api_token():
api_url = os.getenv("PREFECT_API_URL", "")
is_local = api_url.startswith("<http://localhost>") or os.getenv("PREFECT_ENV") == "local"
if is_local:
# local dev: use the local Prefect Block
return Secret.load("my-api-token").get()
# cloud/prod: read from env var or your secret manager
# env var example:
return os.environ["MY_API_TOKEN"]
- Flow usage:
from prefect import flow
@flow
def run():
token = get_api_token()
...
Tips
- Keep the same block name locally and the same env var name in Cloud to simplify code.
- Never commit .env files or scripts with literal secrets.
- If you prefer a secret manager (AWS/GCP/Azure), fetch in the “cloud/prod” branch of get_api_token() using your org’s SDK/role setup. Your code still never stores the secret in Prefect Cloud.
Useful links
- Prefect Cloud UI: Open Prefect Cloud
- Local UI (if running a local server): Open local UI
If you share which secret source you use in Cloud (env vars, AWS Secrets Manager, etc.) and how you run flows (worker type/CI), I can provide a drop-in helper and bootstrap script tailored to that.