Anjaneyulu K

11/10/2022, 7:47 PM
Hi #prefect-cloud, How prefect agent handle the task load for example we are generating one file huge data and it will take 30min. Will Prefect2 agent service wont run out of memory? Can we run the computation outside of prefect agent like running task logic execution on some compute machine and we will just call the call the computation method/function in prefect agent?


11/10/2022, 10:19 PM
How are you running the agent? If you’re running in a VM, the agent will be constrained by the size of the VM. If you’re running in a container in Kubernetes, the agent will be constrained by the resource request/limit settings. Infrastructure blocks provide a flexible way to run flows, which may be in a remote Kubernetes cluster, a service like Cloud Run/ECS — so those are the ways to achieve your goal of running computation outside the agent