https://prefect.io logo
Title
m

Marc Lipoff

02/02/2023, 9:58 PM
Is there a good way to sync blocks and deployments that are in code? For example, delete those not in code, or update those that need updating....
n

Nate

02/02/2023, 10:06 PM
Hi @Marc Lipoff Hmm, what do you mean by "in code"?
m

Marc Lipoff

02/02/2023, 10:07 PM
Eg, I have a example.py file with:
_config_environment_block = JSON(value=_config_environment)
_config_environment_block.save("config", overwrite=True)
and then I usually run
prefect block register -f example.py
but let's say I update this so the name goes from
config
to
config2
.... there exist 2 blocks... whereas I'd really want the first block deleted
n

Nate

02/02/2023, 10:15 PM
Ahh I see so the
prefect block register ...
command is meant to tell your orion server (cloud, or hosted) that the implementation of the
Block
subclass has changed. So you'd only need to
register
if you had done something like added an attribute to
JSON
like
class JSON(Block):
   # existing attrs
   ...
   new_attr: str = "hi!"
the
.save()
method is all you need to save new block instances I'm not sure we'd want orion to inspect your code to see if blocks are being used, but it sounds like maybe a more flexible way to bulk-delete block instances would be useful?
m

Marc Lipoff

02/02/2023, 10:17 PM
ya that would be good. that said, is there a way list all the blocks defined in code? Like a
prefect block register...
dryrun ?
👍 1
n

Nate

02/02/2023, 10:57 PM
when you say
list all the blocks defined in code
do you mean instances of a certain type of block (like
JSON
)? e.g.
_config_environment_block
m

Marc Lipoff

02/02/2023, 10:57 PM
preferably all... but a subset would be fine too
n

Nate

02/02/2023, 10:58 PM
gotcha, I just wanted to make sure you meant instances of a block instead of new block subclass definitions I don't believe there's currently a way to do that, but there might be a simple way that we could support - let me run it by the team
1
looks like its not too bad to do
blocks.py
from prefect.blocks.core import Block

class AddBlock(Block):
    x: int = 1
    y: int = 2
    
    def add(self):
        return self.x + self.y
    
class MultiplyBlock(Block):
    x: int = 1
    y: int = 2
    
    def multiply(self):
        return self.x * self.y
    
a = AddBlock(x=42)
m = MultiplyBlock(y=42)
import inspect
from prefect.blocks.core import Block
from prefect.utilities.importtools import load_script_as_module

filename = "blocks.py" # file where some block instances are defined

# load the file as a module
module = load_script_as_module(filename)

instances = [
    obj for name, obj in inspect.getmembers(module) 
    if not inspect.isclass(obj) and isinstance(obj, Block)
]

print(instances) # yields [AddBlock(x=42, y=2), MultiplyBlock(x=1, y=42)]
i just threw this together but could be formalized if there was a desire to have some sort of dry run as a CLI cmd / util
m

Marc Lipoff

02/06/2023, 7:27 PM
thanks @Nate