Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi Prefect Experts - Im running Prefect 1, but running into an issue that Im having a hard time tracking down.

We have Prefect deployed on Kuberneters and are using Dask Executors. The general pattern is:
users submits a flow
agents creates a job pod
job pod creates a dask cluster (minimum of 1 scheduler and 2 workers)
tasks run on the dask workers

The job pod is consuming memory until it crashes.

Running `htop` on the job pod we can see the prefect process is running `prefect execute flow-run` until all the memory on the physical node is consumed.

I didn’t anticipate much happening on the job pod since we were executing tasks via dask

From the agent
```prefect version
1.4.0```

Hi Bradley, you might find <https://discourse.prefect.io/t/how-does-prefect-send-work-to-a-daskexecutor-and-handle-memory/468|this> discourse article useful in understanding how prefect sends  work to dask and handles memory.

We are certainly using mapped tasks, but we don’t return much data. Almost everything is a pointer to data on S3.