From my understanding, it appears that Prefect 2 only supports script based storage options. Is there any plan for adding built in support for pickle based storage like Prefect 1 had?
a
Anna Geller
01/09/2023, 11:07 PM
pickle proved to be challenging due to issues with portability as it's tied to specific versions and to Python in general
why do you ask?
you totally can solve it without pickle
can you submit an issue with a minimal reproducible example showing why this wouldn't be possible with script storage? Generally, it's not that we are evil by disabling pickle storage 😄 we just saw many problems with it in the past due to portability issues and there doesn't seem to be any use case shown yet that would justify the need where pickle would be strictly required (i.e. no way to solve without pickle)
Anna Geller
01/09/2023, 11:24 PM
a GitHub issue would be better than Slack to share with engineers
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.