Yes — with Prefect 3.x’s Kubernetes work pool/worker you can mount a large, per-run ephemeral volume by customizing the work pool’s base job template to use a generic ephemeral volume (ephemeral.volumeClaimTemplate). Kubernetes will dynamically provision a PVC/PV (e.g., an EBS volume on EKS) for the pod and delete it when the pod is deleted, so each flow run gets a fresh large volume that’s automatically cleaned up. What to do - Use a Kubernetes StorageClass that provisions EBS (via the AWS EBS CSI driver), e.g., gp3. - Extend your work pool’s base job template to add: - spec.template.spec.volumes[].ephemeral.volumeClaimTemplate - spec.template.spec.containers[0].volumeMounts[] - Optionally set spec.ttlSecondsAfterFinished to clean up the Job/Pods soon after completion. Quick way to start 1) Export the default template, edit, then reapply:

prefect work-pool get-default-base-job-template --type kubernetes --file k8s-template.json

In k8s-template.json, add under spec.template.spec: - volumes (ephemeral EBS via PVC template)

"volumes": [
  {
    "name": "ephemeral-storage",
    "ephemeral": {
      "volumeClaimTemplate": {
        "metadata": {
          "labels": { "type": "ephemeral-storage" }
        },
        "spec": {
          "accessModes": ["ReadWriteOnce"],
          "storageClassName": "gp3",               // your EKS StorageClass
          "resources": {
            "requests": { "storage": "200Gi" }     // size you need
          }
        }
      }
    }
  }
]

- container volumeMounts (mount path inside the job container)

"containers": [
  {
    "name": "prefect-job",
    ...
    "volumeMounts": [
      { "name": "ephemeral-storage", "mountPath": "/mnt/work" }
    ]
  }
]

- optional cleanup (delete Jobs/Pods after N seconds)

"ttlSecondsAfterFinished": 300

2) Apply the updated template to a work pool:

prefect work-pool create "eks-pool" --type kubernetes --base-job-template ./k8s-template.json
# or update an existing pool
prefect work-pool create "eks-pool" --base-job-template ./k8s-template.json --overwrite

Notes and tips - This approach uses Kubernetes generic ephemeral volumes so the PVC/PV is created for the pod and deleted automatically when the pod is deleted; no manual cleanup is needed. - Ensure the EKS cluster has the AWS EBS CSI driver installed and you reference a valid StorageClass (e.g., gp3) with dynamic provisioning. - Access mode must match the StorageClass capabilities (EBS is typically ReadWriteOnce). - If you only need small, node-local scratch space, you could use emptyDir instead — but for “very large” storage on EKS, a dynamically provisioned EBS volume via the ephemeral volume template is the right fit. If you can share: - Your StorageClass name, - Desired size (Gi/Ti), - Desired mount path, I can tailor a ready-to-paste base job template for your pool.