YD

    YD

    1 year ago
    Task do not run in parallel I might not understand the parallel execution well. I have two tasks that are not demanded on each other, when I trigger the flow manually, one task runs and the other is pending (when looking at the UI flow run schematic). the code looks like
    from datetime import timedelta
    from prefect import task, Flow
    from prefect.schedules import CronSchedule
    from time import sleep
    
    
    @task(max_retries=1, retry_delay=timedelta(minutes=10), timeout=2000)
    def tast_1():
        sleep(10)
        print('Do task 1')
        return True
    
    
    @task(max_retries=1, retry_delay=timedelta(minutes=10), timeout=2000)
    def tast_2():
        sleep(10)
        print('Do task 2')
        return True
    
    
    def main():
        schedule = CronSchedule("0 15 * * *")
    
        with Flow("parallel tasks", schedule=schedule) as flow:
            r1 = tast_1()
            r2 = tast_2()
    
        flow.register(project_name="parallel tasks")
    
    
    if __name__ == "__main__":
        main()
    Kyle McChesney

    Kyle McChesney

    1 year ago
    What executor are you using? I am not sure what the default is, but the following should enable parallel task execution:
    from prefect.executors import LocalDaskExecutor
    
    ...
    
    with Flow("parallel tasks", schedule=schedule, executor=LocalDaskExecutor()) as flow:
    YD

    YD

    1 year ago
    Thanks, that solved the issue
    Kevin Kho

    Kevin Kho

    1 year ago
    The default is a LocalExecutor which is just single threaded and sequential execution.