Hi all, we are seeing the error "At least one upstream state has an unmappable result" for a failed ...

Paul Vanetti

01/20/2021, 1:35 PM

Hi all, we are seeing the error "At least one upstream state has an unmappable result" for a failed task. A subsequent manual run of the same flow finishes without error. We think this might only happen when the upstream task has to retry several times before succeeding, but I've failed to reproduce this. Any idea what might be causing this?

Jim Crist-Harif

01/20/2021, 4:55 PM

This error should occur if any input to a mapped task is unmappable. For example:

Copy code

task.map([1, 2, 3], 1)  # the 2nd parameter isn't mappable

If all inputs to mapped tasks should be mappable, then there might be a bug (or something else going on). We'd need a reproducible example to help debug further.

Paul Vanetti

01/21/2021, 9:52 PM

Ok I think I have a minimal reproduction:

Paul Vanetti

01/21/2021, 9:53 PM

import datetime

import prefect

@prefect.task(max_retries=10, retry_delay=datetime.timedelta(seconds=900))

def task_a():

if prefect.context.task_run_count < 2:

raise Exception()

return 'a result'

@prefect.task

def task_b(a, c):

return 'b result'

with prefect.Flow('flow name') as flow:

a = task_a()

b = task_b.map(prefect.unmapped(a), [1, 2, 3])

flow.run_config = prefect.run_configs.DockerRun()

flow.storage = prefect.storage.Docker(

# python image

flow.register('project')

Paul Vanetti

01/21/2021, 9:54 PM

I've only done pretty limited testing, but it seems important that the waiting time is long (somewhere at or above 10-15 minutes) and that it uses docker storage.

Jim Crist-Harif

01/21/2021, 10:54 PM

There is a separate code path for retries > 10 min (the flow run should stop and be resubmitted to pick up again later) so that makes sense. I'll open an issue from this thread, thanks.

Jim Crist-Harif

01/21/2021, 10:55 PM

@Marvin open "Mapped tasks fail if an upstream task has a retry > 10 min"

Marvin

01/21/2021, 10:55 PM

https://github.com/PrefectHQ/prefect/issues/3990

Paul Vanetti

01/22/2021, 9:03 AM

Thanks, Jim. I'll keep an eye on this issue.

7 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.