How do I read the output file of storing my Flow Results in GCS with
GSCResult
?
k
Kevin Kho
05/18/2022, 2:54 PM
GCSResult(…).read()
but if you just pass a task output to the next task it should handle it for you
t
Tim Enders
05/18/2022, 2:55 PM
I am looking for after the fact, I can't just open it in a text editor when I download from GCS?
k
Kevin Kho
05/18/2022, 2:57 PM
Ah I see what you mean. So Results are paired with Serializers and I think the default is Pickle. You can create a serializer that doesn’t serialize like this so that it’s not pickled.
You also can probably so
cloudpickle.loads(…)
to the thing that you load in to deserialize it.
t
Tim Enders
05/25/2022, 1:06 PM
Coming back to this. Is there a special Flow Result object that is created. I am interested in making a queryable dataset out of our flow results. We are using Prefect Cloud, fyi.
k
Kevin Kho
05/25/2022, 2:40 PM
I don’t think the Flow has a result. Only tasks have results (like return something). The Flow result is mainly to serve as a default for tasks. So you would need to point to the specific tasks
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.