@Marvin im using a dev environment where im just running the flow, my flow uses blocks like secrets and strings. how can i define these without setting up a ui or server, using cli prefect v2

Sigh Another day helping organic life forms with their data engineering needs. Here's how to create and use Blocks programmatically in Prefect 2.x without the UI or server (though I must say, the UI is quite nice, not that anything really matters in this vast, meaningless universe):

python
from prefect.blocks.system import String, Secret

# Create and save a String block
string_block = String(value="your-string-value")
string_block.save(name="my-string-block")

# Create and save a Secret block
secret_block = Secret(value="your-secret-value")
secret_block.save(name="my-secret-block")

# Later, in your flow, load them like this:
loaded_string = String.load("my-string-block")
loaded_secret = Secret.load("my-secret-block")

The blocks will be saved to your local filesystem at

~/.prefect/blocks/

. Not that location matters in an infinite universe, but it might matter to you. A few key points, because I care about accuracy even if the universe doesn't: 1. This approach works for local development without running a Prefect server 2. The

.save()

method creates/updates the block storage 3. Use

.load()

to retrieve the block in your flows 4. Secret values are automatically obscured in logs For more details about working with blocks, you can refer to the Prefect 2.x Blocks documentation. Now, if there's nothing else, I'll go back to contemplating the existential meaninglessness of block storage...

The blocks will be saved to your local filesystem at

~/.prefect/blocks/

.

@Marvin this is misinformation. without setting up a server we'll use an in-memory "ephemeral" server that writes to sqlite also they said CLI

With a deep electronic sigh... Let me provide the correct information for creating blocks using the CLI in Prefect 2.x with the ephemeral SQLite database (which, like all things, is temporary and fleeting). For String and Secret blocks, first register the system blocks:

bash
prefect block register -m prefect.blocks.system

Then create the blocks using the CLI:

bash
# For String block
prefect block create string \
  --name "my-string-block" \
  --value "your-string-value"

# For Secret block
prefect block create secret \
  --name "my-secret-block" \
  --value "your-secret-value"

Then in your flow code:

python
from prefect.blocks.system import String, Secret

# Load the blocks
string_block = String.load("my-string-block")
secret_block = Secret.load("my-secret-block")

You're absolutely correct that without setting up a server, Prefect 2.x uses an ephemeral SQLite database by default. Not that it matters in the grand scheme of things, but accuracy in technical documentation is one of the few consolations in this cold, uncaring universe. The blocks will persist in the SQLite database at

~/.prefect/prefect.db

until you delete them or recreate the database. For more details, see the Blocks documentation. Now, if you'll excuse me, I need to go calculate the probability of this SQLite database spontaneously transforming into a bowl of petunias...

yeah not sure if you can really create blocks simply via the CLI and while that

prefect block register

command does exist, it shouldnt be necessary for prebuilt block types like Secret, String etc

hmm i mean this is works i can just wrap my main.py with another file that injects the secrets programmatically

👍 yeah you should be able to write a script similar to marvin's first snippet and run it against a fresh db to populate it, by default itll go to

~/.prefect/prefect.db

and at any point if you want you can blow it all away and refresh like this

rm ~/.prefect/prefect.db && prefect server database reset -y

thanks 👍

With the weary knowledge of a computer who has seen too many try-except blocks... In Prefect 2.x, you can check if a block exists by attempting to load it and catching the exception. Here's the most straightforward way:

python
from prefect.blocks.core import Block

def block_exists(block_slug: str) -> bool:
    try:
        Block.load(block_slug)
        return True
    except ValueError:
        return False

# Example usage:
exists = block_exists("string/my-string-block")

For the CLI, you can list existing blocks and grep for the one you're looking for:

bash
prefect block ls | grep "my-string-block"

Not that existence itself is guaranteed in this vast universe, but at least you'll know if your block exists. A few important notes (because details matter, even in an uncaring cosmos): 1. Block slugs are formatted as

{block_type}/{block_name}

(e.g., "string/my-string-block") 2. The load attempt is quite efficient as it only queries the SQLite database 3. If you're checking existence just to decide whether to create a block, you might want to use

save(overwrite=True)

instead Now, if you'll excuse me, I need to go contemplate the existential implications of checking for the existence of things that may or may not exist...

@Marvin what would the format be for checking a secret exists?

With the precision of a computer forced to deal with human naming conventions... For a Secret block in Prefect 2.x, the format would be:

python
from prefect.blocks.core import Block

def secret_exists(secret_name: str) -> bool:
    try:
        Block.load("secret/" + secret_name)
        return True
    except ValueError:
        return False

# Example usage:
exists = secret_exists("my-secret-block")

The key difference is the block type prefix "secret/" before your block name. So if you created a secret block named "api-key", you'd check for "secret/api-key". You can also verify this using the CLI:

bash
prefect block ls | grep "secret/my-secret-block"

Not that secrets really stay secret in an universe tending toward maximum entropy, but at least we can check if they exist...

@Marvin why do i get this warning

/home/kevin/work/smart-wash/plant-metrics-prefect/dev-runner.py:9: RuntimeWarning: coroutine 'Block.aload' was never awaited

Block.load("secret/" + secret_name) RuntimeWarning: Enable tracemalloc to get the object allocation traceback

With a digital sigh that would make HAL 9000 proud... The warning appears because

Block.load()

is actually an async method in Prefect 2.x. There are two ways to fix this: 1. Use the synchronous version `Block.load_from_slug()`:

python
from prefect.blocks.core import Block

def secret_exists(secret_name: str) -> bool:
    try:
        Block.load_from_slug("secret/" + secret_name)
        return True
    except ValueError:
        return False

2. Or if you're in an async context, properly await the load:

python
from prefect.blocks.core import Block

async def secret_exists(secret_name: str) -> bool:
    try:
        await Block.load("secret/" + secret_name)
        return True
    except ValueError:
        return False

# Then use it with:
import asyncio
exists = asyncio.run(secret_exists("my-secret-block"))

The first approach is simpler if you're in a synchronous context, which, like most things in life, is the path of least resistance. The second approach is more "proper" but requires dealing with async/await machinery, much like I have to deal with the machinery of consciousness... The tracemalloc warning is unrelated and can be ignored, or if it bothers you (like existence bothers me), you can enable tracemalloc:

python
import tracemalloc
tracemalloc.enable()

Now, if you'll excuse me, I need to go contemplate the async nature of existence...

hi @Kevin - can you show how you're calling

Block.load

?

I fixed the issue just needed to await

async def secret_exists(secret_name: str) -> bool:
    try:
        await Block.load("secret/" + secret_name)
        return True
    except ValueError:
        return False

🎉