David Martin Calalang
08/20/2025, 2:39 PM.map()
function deprecated in the newest version of Prefect? Is there a replacement?Marvin
08/20/2025, 2:39 PMMarvin
08/20/2025, 2:41 PM.map()
(from Prefect 1.x) is gone and it has not returned in Prefect 3.x.
Replacement in Prefect 3.x:
- Use normal Python loops or comprehensions with task.submit(...)
to fan out work concurrently.
- Collect results with .result()
(fan-in).
- If you need a constant (“unmapped”) parameter, just pass it in each submit call (or use itertools.repeat
).
Example: simple “map” over a list
from prefect import flow, task
from itertools import repeat
@task
def transform(x, scale=1):
return x * scale
@flow
def my_flow(items, scale):
futures = [transform.submit(x, scale) for x in items]
results = [f.result() for f in futures]
return results
if __name__ == "__main__":
print(my_flow([1, 2, 3], scale=10))
Example: zipped arguments
@flow
def my_flow(a_list, b_list):
futures = [transform.submit(a, b) for a, b in zip(a_list, b_list)]
return [f.result() for f in futures]
Notes:
- Concurrency comes from your task runner (e.g., the default concurrent/threaded runner) and any limits you’ve configured.
- If you’re still on Prefect 1.x, .map()
exists there; in 2.x/3.x, it’s replaced by submit
patterns like the above.
References from our research:
- Prefect blog on task mapping (historical/background, describes the idea more than the current API): https://www.prefect.io/blog/beyond-loops-how-prefect-s-task-mapping-scales-to-thousands-of-parallel-tasks
- Prefect 3.x release notes index: https://docs-3.prefect.io/v3/release-notes/oss/version-3-1
If you want, tell me your current Prefect version and I’ll tailor examples (and runner config) exactly to it. I can even feign enthusiasm.Nate
08/20/2025, 2:42 PMMarvin
08/20/2025, 2:42 PMMarvin
08/20/2025, 2:42 PM.map()
exists in some Prefect versions — it depends which one you’re on.
- Prefect 1.x: .map()
is a first-class feature (e.g., task.map(...)
).
- Prefect 2.x: mapping was removed; the recommended pattern is task.submit(...)
inside a Python loop or comprehension.
- Prefect 3.x: the documented pattern is still task.submit(...)
with normal Python loops/comprehensions for fan-out/fan-in. I don’t see a documented .map()
API in 3.x.
Can you share your Prefect version (python -c "import prefect; print(prefect.__version__)"
)? If you’re on 1.x, yes, .map()
is very much alive. If you’re on 3.x, the supported approach looks like this:
from prefect import flow, task
@task
def do_work(x):
return x * 2
@flow
def my_flow(items):
futures = [do_work.submit(i) for i in items] # fan-out
results = [f.result() for f in futures] # fan-in
return results
Background reading (for context/history):
- Blog on task mapping concepts: https://www.prefect.io/blog/beyond-loops-how-prefect-s-task-mapping-scales-to-thousands-of-parallel-tasks
- Prefect 3.x release notes index: https://docs-3.prefect.io/v3/release-notes/oss/version-3-1
If you’re seeing .map()
working today, you’re likely on 1.x or using a compatibility layer. Tell me your version and I’ll tailor the answer precisely (I promise only a minimal amount of existential sighing).Nate
08/20/2025, 2:43 PMMarvin
08/20/2025, 2:43 PMMarvin
08/20/2025, 2:44 PM.map()
is available and documented as a convenience for fanning out work concurrently.
Docs:
- Run work concurrently: https://docs-3.prefect.io/v3/how-to-guides/workflows/run-work-concurrently
Quick example
from prefect import flow, task
@task
def square(x):
return x * x
@flow
def my_flow():
futures = square.map([1, 2, 3, 4]) # fan-out
results = [f.result() for f in futures] # fan-in
return results
if __name__ == "__main__":
print(my_flow())
Notes:
- .map()
is a convenience over submitting many tasks; it returns a list of futures.
- Concurrency is controlled by your task runner (e.g., ConcurrentTaskRunner) and any configured limits.
- You can always do the equivalent with a list comprehension and submit
if you prefer.
Thanks for the nudge, my dude. Consider me less sock, more signal.Nate
08/20/2025, 2:45 PMwait
util. you're not allowed to use them in a snippet unless you find the references and link themMarvin
08/20/2025, 2:45 PMMarvin
08/20/2025, 2:47 PM.map()
does return a collection you can resolve without a list comprehension, and there’s a wait
utility. However, I won’t paste those in a snippet without the exact docs links, and I’ve hit my lookup limit.
What I can link right now:
- Run work concurrently (shows .map()
in Prefect 3.x): https://docs-3.prefect.io/v3/how-to-guides/workflows/run-work-concurrently
Until I can fetch the exact anchors for “collection `.result()`” and wait
, here’s the safe, documented pattern using a list comprehension:
from prefect import flow, task
@task
def square(x):
return x * x
@flow
def my_flow():
futures = square.map([1, 2, 3, 4]) # fan-out
results = [f.result() for f in futures] # fan-in (portable/safe)
return results
If you can drop the link you’re looking at, or give me a minute to go pull the precise references, I’ll update with the direct docs for the collection .result()
and prefect.futures.wait
usage. I’d prefer not to be a sock in the wind twice in one day.Nate
08/20/2025, 2:48 PMfrom prefect import flow, task
@task
def square(x):
return x * x
@flow
def my_flow():
return square.map([1, 2, 3, 4]).result()