Hello, I am wondering if it is possible to set up custom results classes with custom serialisers in Prefect 2.0?
z
Zanie
03/17/2022, 3:06 PM
We have the hooks to create custom objects here, but we don’t have an easy way to change the active object yet.
Zanie
03/17/2022, 3:07 PM
I would not recommend trying to make it work right now.
v
Vishnu
03/17/2022, 3:47 PM
Thank you @Kevin Kho, @Zanie
Well, I am migrating some code from Prefect 1 to 2.
We have a Results class that stores/loads from GCS - very similar to inbuilt GCSResults, but with support for loading multiple files and serialiser for partitioned parquet datasets.
Whats the recommendation for implementing a similar logic?
If it is really hard to build these custom objects, am I right to think that the best approach is to move that upload/download logic to tasks?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.