Jullian Bellino
04/04/2022, 8:38 AMraise RETRY("Partition not available yet, retrying in 1 hour.", start_time=pendulum.now().add(hours=1))
However it looks like this could block the other running tasks because the runner is not released. Now let's imagine we only rely on a single local agent, with maximum 3 tasks in parallel, in case some S3 partitions are late for any reason, it could quickly block all of the others tasks. Do you have any better way to achieve this ?Anna Geller
04/04/2022, 9:26 AMJullian Bellino
04/04/2022, 9:49 AMAnna Geller
04/04/2022, 10:43 AM