Florian Laws
01/16/2025, 12:55 PMMarvin
01/16/2025, 12:55 PM@task
decorator to define tasks and the @flow
decorator to manage the flow of tasks. Here's a concise way to map over a list, process items with one task, and then process each result with the next task:
1. Define Tasks: Use the @task
decorator to define each processing step as a task.
2. Map and Chain Tasks: Use the map
method to apply a task to each item in a list. Then, chain the tasks by using the output of one task as the input to the next.
Here's a simple example:
python
from prefect import flow, task
@task
def process_item(item):
# Your processing logic here
return item * 2
@task
def further_process(result):
# Further processing logic
return result + 1
@flow
def process_data(data):
# Map over the list
processed_items = process_item.map(data)
# Further process each result
final_results = further_process.map(processed_items)
return final_results
# Example usage
data = [1, 2, 3, 4]
results = process_data(data)
This code sets up a data processing pipeline where process_item
is applied to each element of data
, and then further_process
is applied to each result. The map
method is key to applying tasks over lists in Prefect.
For more details, you can refer to the Prefect tasks documentation.