<@ULVA73B9P> I would like to move blocks from one ...
# ask-marvin
t
@Marvin I would like to move blocks from one workspace to another. Is there a script based way to do so?
m
thought for 680.4 seconds
Short answer: Yes—there isn’t a one-click “migrate blocks” command, but you can script it. In Prefect 3.x you can load a block from one workspace and save it into another using the Python API. Two quick notes for your setup: - Since you’re self-hosting with server.api.auth_string, set
PREFECT_API_AUTH_STRING
for clients (or pass
auth_string=
to PrefectClient). Make sure
PREFECT_API_KEY
is NOT set, as it will take precedence and break auth for Server. See Security settings. - If your workspaces are on different API URLs, just use different
api=
values. If they’re on the same API URL but different workspaces, use whatever method you already use today to target a workspace (URL path vs header). If that’s unclear, I can help with the exact client config—just let me know how you currently select a workspace from code/CLI. Scripted copy by block slug - This works well from Jupyter and preserves secrets. - It assumes the target workspace has the same block type registered (for built-in/integration blocks). For custom blocks, register them in the target first (example CLI shown below).
Copy code
import asyncio
from prefect.client.orchestration import PrefectClient
from prefect.blocks.core import Block

async def copy_blocks(slugs, src_api, src_auth, dst_api, dst_auth, overwrite=False, validate=True):
    async with PrefectClient(api=src_api, auth_string=src_auth) as src, \
               PrefectClient(api=dst_api, auth_string=dst_auth) as dst:
        for slug in slugs:
            # slug format: "<block_type_slug>/<block_name>"
            print(f"Copying {slug} ...")
            block = await Block.load(slug, validate=validate, client=src)
            block_name = slug.split("/", 1)[1]
            await block.save(name=block_name, overwrite=overwrite, client=dst)
    print("Done.")

# Example usage:
# Fill these in for your source/target workspaces
SRC_API = "<https://your-src-prefect/api>"
SRC_AUTH = "your-src-auth-string"
DST_API = "<https://your-dst-prefect/api>"
DST_AUTH = "your-dst-auth-string"

# List the slugs you want to move (from `prefect block ls` in the source)
SLUGS = [
    "s3-bucket/raw-data",
    "aws-credentials/prod",
    "secret/service-token",
]

await copy_blocks(SLUGS, SRC_API, SRC_AUTH, DST_API, DST_AUTH, overwrite=False)
How to list the slugs to migrate - In each workspace, you can list configured blocks:
Copy code
prefect block ls
- To inspect the details of a specific block and confirm the slug:
Copy code
prefect block inspect <BLOCK_TYPE_SLUG>/<BLOCK_NAME>
Registering block types in the target (only if you use custom or integration blocks) - Register the block types your blocks depend on in the target workspace before copying. For example:
Copy code
# Example: register AWS collection blocks
prefect block register -m prefect_aws.credentials
CLI help (verified): -
prefect block --help
-
prefect block register --help
-
prefect block ls --help
-
prefect block inspect --help
Advanced/fully generic approach - If you want to migrate “everything” without pre-listing slugs, you can iterate with the Python client: -
PrefectClient.read_block_documents(include_secrets=True)
to list block docs in the source, then for each, load via
Block.load("<type>/<name>", client=src)
and
save(...)
into the target. - This still relies on the target workspace having matching block types/schemas. If not, register them first. A couple of gotchas - If a block of the same name already exists in the target, set
overwrite=True
in
save(...)
or delete it first. - If schemas have drifted (e.g., old block versions), you can pass
validate=False
to
Block.load
to be more lenient. - Do not log or print block contents that include secrets. Docs and references - Security and auth string: Security settings - Workspaces overview: Workspaces
- Blocks concepts: Blocks Open question to tailor this for you - Are the two workspaces on the same Prefect Server instance or on different servers/base URLs? - How are you currently selecting a workspace from code (different API URLs vs a header)? If you share that, I’ll adapt the client setup in the script precisely for your environment.
b
We do now have a migration tool, but it will currently migrate everything in your account. I believe there may be some enhancements in the future for limiting migrations to specific resource types like Blocks. https://docs.prefect.io/v3/how-to-guides/migrate/transfer-resources
I went ahead and opened an issue to track this: https://github.com/PrefectHQ/prefect/issues/18788
🎯 1
t
Hey @Brendan Dalpe that's very cool. I didn't know about the prefect transfer tool. In the end I built a throwaway script that looped through all the blocks and then saved them to the new account.
1