Hello Team - After storing/persisting the results ...
# ask-community
s
Hello Team - After storing/persisting the results of the flows/tasks with the configuration persist_result=True and result_storage=s3_storage_block, how can these results be retrieved/managed for later consumption? Earlier, the endpoint
artifacts/filter
was returning the flowrun metadata and that contains the storage location information. But it is no longer the case now (maybe because results are not artifacts). Can someone help here, please?
Could anyone answer this question, please?
n
hi @SantanM you can interact with the results in your result storage directly (ie using aws cli, boto or otherwise) or you can read the result by storage key like this
Copy code
import asyncio

from prefect import task
from prefect.results import ResultStore


@task(
    persist_result=True,
    result_storage_key="my-storage-key",
)
def add_em_up(x: int, y: int) -> int:
    return x + y


async def main():
    add_em_up(42, 42)

    record = await ResultStore().aread("my-storage-key")
    assert record.result == 84, "expected a result of 84"
    print(record.model_dump_json(indent=2))


if __name__ == "__main__":
    asyncio.run(main())
Copy code
» python sandbox/proofs/load_saved_result.py
23:59:33.876 | INFO    | Task run 'add_em_up' - Finished in state Cached(type=COMPLETED)
{
  "metadata": {
    "storage_key": "/Users/nate/.prefect/storage/my-storage-key",
    "expiration": null,
    "serializer": {
      "type": "pickle"
    },
    "prefect_version": "3.1.10.dev2",
    "storage_block_id": null
  },
  "result": 84
}
s
I wonder why results are separated from flowrun/taskrun. Based on your answer, prefect does not preserve any relationship between flowrun/taskrun and its result. What can be a good reason for that?
Thanks @Nate for answering my first question. I’ve a followup on that - In my case, there is a HTTP client seeking the result of a flowrun (after the completion of a flowrun). Is there a way to fetch result through a REST endpoint using the
flowrun id
?
n
results are not stored on the server, they're stored client side (i.e. in your storage), so there is not a REST endpoint for fetching results there are many reasons for this, the main one is that results are often sensitive
s
there are many reasons for this, the main one is that results are often sensitive
Yes, I understand this @Nate In Prefect v2,
artifacts/filter
endpoint was returning (S3) location of the generated result of a flowrun/taskrun. It seems in Prefect V3 this behaviour has changed and the response from the endpoint does not include result storage location information. Is there a way to retrieve result storage information (eg: S3 bucket path) of a flowrun/taskrun ?
n
can you open a discussion detailing the change in behavior you've noticed?
👍 1
s
@Nate - here is the discussion thread on the issue. Please let me know if you need more details