Hi All,
I use the persist_result for my tasks, however I have two agents and when retrying flow it may happen that the flow no longer runs on the same agent. So I can't use local storage (PREFECT_HOME/storage)
I don't want to use s3, is it possible to use prefect's blocks to store these results? or even better, use the postgres DB? Thanks
d
Dominic Tarro
06/17/2023, 8:08 PM
Why would Prefect Blocks be better than S3? Either way you get slowed down by data transfer.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.