Yardena Meymann

08/07/2022, 7:09 AM
Hi, I am using Prefect 1.2.1, how can I obtain the location of the result of the previous task (that uses GCSResult) - I want to pass the location of the data, not the data itself to the next task?

Anna Geller

08/07/2022, 12:37 PM
Many ways to solve it: 1. The easiest would be if you would return the path e.g. as a string and pass it to the downstream task as data dependency 2. Do both in your task: write to GCS and pass data itself to a downstream task that needs it 3. Use templating so that you know exactly the path:
from prefect.engine.results import GCSResult

gcs_result = GCSResult(bucket='prefect_results', location='{flow_name}__{task_name}_xxx.pickle')

def my_task():
and just to highlight: you don't have to use GCSResult to write data to GCS, in fact I'd encourage you to build custom logic; results are mostly for recovery from failure/retry/caching and orchestration, less so for actual writing of data

Yardena Meymann

08/07/2022, 1:27 PM
Thanks @Anna Geller, got it!
🙌 1