K8s provides a nice interface for deploying a dynamic number of Jobs that are easy to maintain, watch, delete, etc… and it’s a tool I’ve been familiar with for a while. It allows you to run the Prefect k8s agent inside your cluster and when it finds flows to run it will kick them off right there in the cluster. Then it has all the added benefits of autoscaling to meet demands.
Something like Fargate also provides some nice scaling and modularity but I have much more experience working with k8s!