Has block.load(<name>, validate=True/False) been removed? I see in the docs

my_s3_bucket_block = S3Bucket.load("my-s3-bucket", validate=False)

but upon trying to execute this code, i am seeing

Block.load() got an unexpected keyword argument 'validate'

Hi @Matthew Scanlon, I recommend using the S3Bucket from the prefect-aws collection. Where in the docs do you see that example?

I am just coping the example from the docs. In practice it is a custom block i am writting

hi @Matthew Scanlon, the

validate

kwarg was added to

Block.load

quite recently, what version of prefect are you on?

2.7.7

hmm I am not able to reproduce that by following those steps, do you mind sharing the output of

prefect version

from where you encounter the error?

Thanks for the suggestion, this actually lent it-self to the realization i was running the script in the wrong venv. Updating working like a charm. On a related note. Does prefect have any guidance for version control of custom blocks. It seems like the UI does not update block fields given the underlying block is re-registered (updated) with different fields. This means we would have to programmatically change fields, delete and recreate blocks, or constantly load with

validate=False

which lends itself to a plethora of other issues. I would imagine there has to be some way in which you guys are managing changes made to your own "official" blocks

It seems like the UI does not update block fields given the underlying block is re-registered (updated) with different fields.

one thing that might be helpful to note is that when you call

prefect block register -m some_module

, you're not registering a single block instance, you're registering a type of block (whatever

class Foo(Block):

lives in file / module you pass to the

block register

command the purpose of the

validate

kwarg accepted by

Block.load

is just to migrate existing block instances from one version of a block type to another, so for example if I have

from prefect.blocks.core import Block

class Foo(Block):
   x: int = 1

foo = Foo()
foo.save("testfoo")

then, as you'd expect, we see (first image) then, simulating a change to the block implementation:

from prefect.blocks.core import Block

class Foo(Block):
   x: int = 1
   y: int = 2

# pydantic validation would fail, bc block instance in
# orion database doesn't know about new local definition of class Foo
# so we use validate=False
foo = Foo.load("testfoo", validate=False) 

assert foo == Foo(x=1, y=2) # passes

# saving with overwrite will re-register the block type for you, 
# if and only if schema has changed, based on the schema 
# of the updated local class Foo (e.g. adding y)
foo.save("testfoo", overwrite=True)

and we should now see (second image)

This was really helpful nate, thank you. What i meant however, was in your example here, if i were to create that first block

testfoo

with x = 1, and the do

prefect blocks register --file my_block_file

on the second instance of class Foo, i am not going to see Y show up in the UI for testfoo. If i try to create a new Foo block then yes, that field is available, but testFoo only has field 1. As a cloud user managing many blocks across many users, I think it is quiet dangerous to have programmers interacting with blocks programmatically. It makes much more sense from my perspective that if i register a block with a schema change, and then go to an old version of that block, the new schema is present and i can update the fields accordingly

This makes it out to sound like blocks are tied to a specific version of a block instance, which they are not

Block documents are in fact tied to a specific version of the block schema, which means that in principle its possible to have block documents associated with different schemas for a given block type. This is why the documentation example shows the migration of a single block document it sounds like you'd like to see a cascade, where if you reregister a block type, all block documents of that type would be conformed to the new schema, but that is not currently our intended behavior

Block documents are in fact tied to a specific version of the block schema

I see, i think this finally clicked for me. So in your example above. TestFoo gets saved with schema 1, but then after re-registering Foo, on Foo.load('TestFoo'), the engine is going to use FooV2 but TestFoo was written with FooV1, so i need to re-write fooV2 in order for the validator to not complain.

I think of blocks as easiest to interact with programmatically, but I am of course biased

I think it is only in the enterprise version of prefect cloud that you can have role based permissions, correct? We have been discussing internally whether upgrading makes sense as right now, it is very easy for anyone on our prefect account to do something like

Secret.load('SomeSecretBlock).get_secret_value()

and see what the token associated with that secret is. This was a pain point in prefect 1 with being able to query the client for secret values. So the idea that we could have a block with an associated secret field, that a programmer can go in and change, and then i can not even log into the ui to see if the value is correct, just seems problematic to me. As i will have to write a script to log into prefect, load the secret, and programmatically compare it/ update it to ensure it is correct. Specifically in my case, not everyone we have writing flows is a software engineer, so expecting them to not only learn how to write flows but then also maintain them programmatically i see as a pain point for our specific use cases. We as programmers are not perfect, mistakes get made all the time, i just think the UI lends it self to being less prone to errors.

its just an escape hatch for when you'd want to avoid recreating an existing block instance after re-implementing the class

i agree, and i discussed this quite extensively with my colleagues, as i have taken a position quite similar to your own. The dict is a great idea for achieving this.

So in your example above. TestFoo gets saved with schema 1, but then after re-registering Foo, on Foo.load('TestFoo'), the engine is going to use FooV2 but TestFoo was written with FooV1, so i need to re-write fooV2 in order for the validator to not complain.

yep exactly!

validate=False

is just to tell pydantic to relax for a moment so we can we update

TestFoo

from

FooV1

to

FooV2

This was a pain point in prefect 1 with being able to query the client for secret values. So the idea that we could have a block with an associated secret field, that a programmer can go in and change, and then i can not even log into the ui to see if the value is correct, just seems problematic to me. As i will have to write a script to log into prefect, load the secret, and programmatically compare it/ update it to ensure it is correct.

this is an interesting point. I believe at the organization pricing tier you get normal RBAC with Owner, Collaborator and Read-Only (enterprise tier you also get to create custom RBAC roles). So if you have an organization, you should be able to to restrict certain people to Read-Only, which would prevent them from Create / Delete / Edit of block instances (including secrets)

Yes. We are at the organization level. It would be nice to some sort of distinction that allows us to have users that can not manually read in secret values. But i would assume that comes with custom RBAC at the enterprise level