https://prefect.io logo
Title
m
r

redsquare

04/06/2023, 3:39 AM
I initially went down a similar route to store checkpoints per flow in json blocks however I was advised by Prefect that there could be concurrency issues. This was 8 months ago so not sure if the stance has changed since. If your flows are not concurrent then this should be ok. I dont like to use exception handling like this, you could check if it exists like this https://github.com/PrefectHQ/prefect/blob/4a1b44a3fcfe628fefc9d3bf8a0cc57768822977/src/prefect/cli/block.py#LL346C32-L346C68
m

Matt Fysh

04/06/2023, 5:40 AM
good point about concurrency, this is intended to be sequential only so I've added that to the flow def, even though sequential is the default (I think?) good to be explicit about it so there are no issues with the state block + concurrency on the exception handling, it seems you are supposed to use ValueError when loading and it indicates BlockNotFound: https://docs.prefect.io/api-ref/prefect/blocks/core/#prefect.blocks.core.Block.load under the Raises section:
ValueError
If the requested block document is not found.
r

redsquare

04/06/2023, 6:08 AM
ok, not seen that - it still never sits easy with me to use exceptions for non exceptional circumstances!
however it looks like EAFP coding style is the pythonic way