Hi everyone! I’m exploring ideas around dynamically configuring a Fargate instance/cluster based on a flow parameter / input.In short, I have an hourly job that migrates data for a group of clients based on their timezone. For some hours of the day, there will be more clients and thus more work. Ideally, I want my flow to scale resources based on the number of clients queried for a timezone.Does anyone have ideas for a way to do this? Since flow environments are set at registration time, I’m struggling to find a feasible approach. I’ve also started to explore the new
recently added, which sound like a potential solution.Thanks for the input!
1 year ago
@Mike Marinaccio We do exactly this using DaskCloudProviderEnvironment and the