https://prefect.io logo
Title
e

Eddie Atkinson

05/02/2022, 5:58 AM
I am currently running my jobs on Fargate but have bumped into their (weirdly low) limits on the number of cores I can use. What would be the easiest upgrade pathway for running jobs provided I wanted to keep using serverless? My guess would be it’s time to switch to a
DaskExecutor
and then use a temporary cluster as described here: https://docs.prefect.io/orchestration/flow_config/executors.html#using-a-local-cluster Just want to make sure I’m not missing a more obvious solution before I move to a cluster.
a

Anna Geller

05/02/2022, 10:18 AM
Hi Eddie, you're right that the lowest number of CPU cores is 4. Moving to FargateCluster dask cloud provider seems like a reasonable approach
e

Eddie Atkinson

05/03/2022, 12:54 AM
Awesome, thanks Anna 🙂
👍 1