Hi all! 🙂 I have a question (maybe it's stupid but here it goes xD) I'm working with Prefect in a Kubernetes environment, which means that the agent and the jobs are running in the cluster. The cluster has 2 CPUs. When I'm trying to run three parallel tasks, only two of them run and then when one of them ends the third one starts running. Is that the expected behavior or I'm doing something wrong? Thanks a lot for your tool, Prefect rocks! 😎

Can you share your flow code? by default, a ConcurrentTaskRunner is used which will run your tasks concurrently (IO operations), but won't parallelize work across CPUs

...

# task 1

set_status_task_1_processing = update_status(

merge_processing_backend, constants.TASK_1

)

task_1_processed = process_task_1(

set_status_task_1_processing

)

set_status_task_1_finished = update_status(

task_1_processed, constants.STATUS_TASK_1_FINISHED

)

# optional task 2

with case(task_2_needed, True):

set_status_task_2_processing = update_status(

merge_processing_backend, constants.TASK_2

)

task_2_processed = process_task_2(

set_status_task_2_processing

)

set_status_task_2_finished = update_status(

task_2_processed, constants.STATUS_TASK_2_FINISHED

)

# optional task 3

with case(task_3_needed, True):

set_status_task_3_processing = update_status(

merge_processing_backend, constants.TASK_3

)

task_3_processed = process_task_3(

set_status_task_3_processing

)

set_status_task_3_finished = update_status(

task_3_processed, constants.STATUS_TASK_3_FINISHED

)

merge_end = merge(

set_status_task_1_finished,

set_status_task_2_finished,

set_status_task_3_finished,

)

you can disable sending all responses to the main channel by not checking this checkbox - could you fix that to keep the main channel clean?

ups, sorry! 😅

no worries, happens to me every now and then too - thanks so much! so you have some quite complex task dependencies with branching logic and this is not well suited for parallel processing - since you're using 1.0, there is a concept of mapping to run things in parallel - can you check if using mapping helps? https://docs.prefect.io/core/concepts/mapping.html

Thanks a lot! 🙂 I'm going to try the mapping then!