Hi Everyone, I have a flow that have mapped tasks ...
# prefect-community
m
Hi Everyone, I have a flow that have mapped tasks that use DaskCloudProvider Environnement and i have set the number of workers to 4. For some reasons only one worker is created. Here is the code:
Copy code
flow.environment = DaskCloudProviderEnvironment(
    provider_class=FargateCluster,
    # task_role_arn="arn:aws:iam::497427061914:role/ecsTaskRole",
    execution_role_arn="arn:aws:iam::497427061914:role/ecsTaskRole",
    n_workers=4,
    scheduler_cpu=256,
    scheduler_mem=512,
    worker_cpu=512,
    worker_mem=1024,
    labels=["fargate"]
)
d
Hi @Mac Gréco Péralte Chéry! I’m not as familiar with the DaskCloudProviderEnvironment, but I would double-check that your FargateCluster has adequate resources to spawn multiple workers
m
Hello @Dylan, to be more precise, when i run my flow i see the fargate prefect task created into my AWS console but it seems that my flow is executed in this task but as far as i remember this first task should instanciate another fargate task that will dynamicaly create a dask cluster. It used to be like that before i updated to the latest version of prefect (0.12.1 ->0.13.15)
I am using an prefect cloud and and Ec2 instance to run my agents