Paulo Benatto

06/17/2021, 8:59 AM
Hi guys, how r u doing? I'm new to prefect and i was reading: and i think im missing something ๐Ÿ˜. I would like to build a list of tasks to be executed in parallel. See this dummy example:
def create_task_list(ids):
    l = []
    for i in ids:
    return l

def generate_ids():
    return ["id1", "id2", "id3"]

with Flow("parallel-execution") as flow:

    ids = generate_ids()
    list_of_tasks = create_task_list(ids)

    request_referral = RequestReferralDetail()

    #ย list_of_tasks should be a list, but i'm returning a task. how to inject a list on bind?
    request_referral.bind(request_details=list_of_tasks, flow=flow)


Rob Fowler

06/17/2021, 9:57 AM, unampped(static_argument)
would run multiple instances of the RequestReferralDetail for each item in list_of_items
although from your language you want want to run multiple different tasks in parallel which happens automagically. My example is running a task multiple times in parallel operating on a list of items to be operated in parallel

Paulo Benatto

06/17/2021, 10:05 AM
lovely @Rob Fowler i will read the page now ๐Ÿ™‡

Rob Fowler

06/17/2021, 10:16 AM
just a ps. If you are wanting more threads you can increase them: flow.executor = LocalDaskExecutor(workers=30, .. I find processes more stable for large numbers of workers but YMMV
๐Ÿ™Œ 1

Paulo Benatto

06/17/2021, 12:44 PM
it does work, however i have to call
method inside of
class RequestReferralDetail(Task):
    def run(self, request_details):
        if isinstance(request_details, Task):
I think I still missing something, probably i will read more doc ๐Ÿ˜„ Thanks ur help mate