Hi, I am using Prefect 1.2.1, how can I obtain the location of the result of the previous task (that uses GCSResult) - I want to pass the location of the data, not the data itself to the next task?
✅ 1
a
Anna Geller
08/07/2022, 12:37 PM
Many ways to solve it:
1. The easiest would be if you would return the path e.g. as a string and pass it to the downstream task as data dependency
2. Do both in your task: write to GCS and pass data itself to a downstream task that needs it
3. Use templating so that you know exactly the path:
and just to highlight: you don't have to use GCSResult to write data to GCS, in fact I'd encourage you to build custom logic; results are mostly for recovery from failure/retry/caching and orchestration, less so for actual writing of data
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.