Hey I'm running a few subflows that call several tasks with azure work pool in prefect v3. My yaml deployment file defines the image platform as
linux/amd64
. For the most part, each of my subflows are working with 10-30 subtasks. When the subtasks increase to ~500, my container crashes with the error message
Failed due to a(n) Container exited with non-zero code (Error Code: 137) likely caused by misspecified platform or architecture
I've tried to increase my workpool resources but it still seems to fail:
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.