Does anyone know how to read a persisted result?
# ask-community
g
Does anyone know how to read a persisted result?
1
g
Thanks! Also, seems like you can achieve this if you have the FlowRun object. E.g.
flow_run.state.result()
n
indeed you can!
g
Just for my edification: I've decorated a flow with
persist_result=True
and
result_storage=GCS(bucket_path="xyz")
and created a deployment. I'm then running that deployment from another deployment where I use
from prefect.deployments import run_deployment
to get the completed FlowRun object. It seems like prefect automatically stores the ref to the bucket when its saved and correctly loads the result from that bucket without having to configure the bucket location anywhere in the parent deployment that is calling run_deployment. Is my understanding correct?
n
I think this should be >
result_storage=GCS.load("some-existing-gcs-block-with-defined-bucket-path")
and once you have that > It seems like prefect automatically stores the ref to the bucket when its saved and correctly loads the result from that bucket without having to configure the bucket location anywhere in the parent deployment that is calling run_deployment. Is my understanding correct? then yes
g
Interesting. Right now I have a flow that looks like this:
Copy code
gcs_storage = GCS(bucket_path="xyz")

@flow(log_prints=True, persist_result=True, result_storage=gcs_storage)
def flow_b():
    return 'some value'
And that gets served as a deployment. Then in another deployment I have a flow that doesn't pass any parameters for persist_result or result storage, but is able to retrieve the result using
flow_run.state.result()
This is a POC before I go further on implementing a larger flow, but I want to make sure I'm not doing anything that would lead to unexpected results.
n
lemme double check my understanding of whats happening there, we might go ahead and save that for you
g
That would be awesome! For your ref, I'm using a parent deployment that will invoke other deployments using
run_deployment
which returns the
FlowRun
object that I plan on using to fetch the results. Those sub-deployments will have result_storage configured to GCS. The parent flow will also make calls to subflows, but we use deployments when the python environments differ so we can maintain dependency isolation across job types and servies
n
yeah i think that should be fine as well, looking at this it seems like we'll save that as an anonymous block if you're planning on using that same GCS bucket path all over the place then I might recommend saving it explicitly yourself so you can see it in the UI or via
prefect blocks ls
but that seems like a matter of preference / use case 👍
🙌 1
g
Beautiful! I haven't fully wrapped my head around the concept of blocks, but are these mostly for credentials? We're self-hosted in GKE and each flow runs as a Kubernetes job using a service account that has access to the infrastructure it needs
n
Blocks are basically fancy pydantic `BaseModel`s you can save in our db as an interface to some 3rd party resource / system, so yeah commonly used for general config / credential management. but they're nice bc you can hang methods off of them (as we do in many of our native block impls or you could do in your own block types) your k8s work pool is where you config infra stuff like image pull policy / service acct stuff for your compute provider @Marvin wanna give a rundown on what prefect blocks are?
m
Prefect blocks are a fundamental concept within Prefect that enable the storage and management of configuration data, and provide an interface for interacting with external systems. Here's a summary of what Prefect blocks offer: - Configuration Storage: Blocks allow you to securely store credentials required for authenticating with various services like AWS, GitHub, Slack, etc. This is critical for orchestrating interactions with these services through Prefect. - Interface for External Systems: Blocks expose methods that give you pre-built functionality to perform actions against an external system. This can include uploading or downloading data from cloud storage, querying or writing data to databases, or sending messages through communication platforms. - Accessibility: You can configure blocks through code or visually via the Prefect Cloud and the Prefect Server UI. - Usage: Blocks can be accessed for configuring flow deployments and can be directly utilized within your flow code. - Built-in Block Types: Prefect comes with several built-in block types for common use cases. These can be used out-of-the-box and additional blocks are available through Prefect Integrations. - Library of Block Types: Prefect Cloud and the Prefect Server UI provide a library of block types that you can configure for use by your flows. - Parameters vs. Blocks: Blocks are meant for configuration that needs to be consistent across flow runs and shared among multiple flows. For configuration that varies between flow runs, it's recommended to use parameters. To learn more about Prefect blocks, visit the official documentation.
g
Gotcha. Thanks, will look into for general config and parameter management. This has been super helpful — much appreciated @Nate 🙏
n
no prob catjam