https://prefect.io logo
Title
l

Luke Orland

02/10/2021, 10:15 PM
The recently updated graphic makes it look like the mapped tasks are not being shared among the workers. The long horizontal green lines are the two mapped tasks spawning dozens of tasks copies each. Or does this visualization show that the mapped tasks actually running in series on a single worker?
Flow(
  flow_name,
  run_config=DockerRun(),
  executor=LocalDaskExecutor(
    scheduler="threads",
    num_workers=8,
  ),
)
n

nicholas

02/10/2021, 10:21 PM
Hi @Luke Orland - that chart doesn't show mapped task runs as individual nodes - mapped task bars like that are meant to encapsulate the entirety of the spawned task run children but not to represent their parallelism (or lack thereof)
This is mostly due to space constraints in that chart crossed with the potential scale of mapped pipelines (which can get into the hundreds of thousands of runs)
l

Luke Orland

02/10/2021, 10:23 PM
makes sense!
👍 1