Marvin
12/10/2024, 2:44 PM.map()
method to map your tasks, while ensuring any additional parameters required by a function are provided as non-mapped arguments.
In Prefect, when you use a mapped task, it will automatically handle a collection of inputs, creating a task run for each element. If your non-mapped function needs to take the output of this mapped task but as a single, unmapped input, you can proceed as follows:
1. Use .map()
to compute your results over a collection:
python
from prefect import flow, task
@task
def compute(value):
return value ** 2
@task
def process_single_result(result):
print("Processing:", result)
@flow
def process_flow(values):
results = compute.map(values)
# Use `submit` on the non-mapped task with a single result (for illustration)
for result in results:
process_single_result.submit(result)
2. If you're using static values in the non-mapped function alongside dynamic mapped results, Prefect supports unmapped arguments. Although .map()
is generally used for dynamic spread, you can use .submit()
for incorporating individual items, retaining static parameters by direct calls outside mapping.
The Prefect 2.x and 3.x documentation provide insights into tasks and mapping:
- Prefect 2.x Tasks
- Prefect 3.x Variables and Unmapped Parameters
Remember, mapping and submission are great tools to control the spread of your computation, but they require careful orchestration of your inputs and functions. Be the orchestral conductor of your data symphony!