Hi, I am currently running prefect flows using ECS...
# prefect-integrations
Hi, I am currently running prefect flows using ECSTask infrastructure block setup, in an ECS Fargate cluster, it takes around 3-4 minutes for a flow to start, does anyone have some advice on how to speed up the process?
how large are the images
I would look into trying to make the images that the cluster is grabbing smaller.
fargate has to spin up the ecs block and pull the image, this isn’t unusual
👍 1
image is 3+ GB
I guess my problem is that I use this image for all the flows then?
I have flows that take less to run then the spin-up.. But also flows that run for hours for ML.. My thoughts are to move the small flows from prefect to just lambda functions.
If your different flows have pretty dissimilar dependencies, it might make sense to have an image for each flow (or group of flows with similar dependencies) - as mentioned above, it kinda makes sense to me that a 3GB+ image would take a bit to pull and start up on fargate