Let me ask my question better:
I appear to be hitting new errors where task results cannot be persisted because the inputs are not pickle-able to make a cache key.
• Question 1 - this seems to be new behavior. Did something change where task results are persisted by default (I did not set any retry behavior and I don't want/expect these tasks to be rerun at this point, though I hope to add that functionality in the future)
• Question 2 - I see an argument (
results_serializer
) to create a serializer for my task's outputs, but in order to make the inputs "pickle-able" for a retry attempt, I would need an
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.