@Marvin I have a flow that runs a subprocess via python subprocess, which requires GDAL to be installed (specifically uses

ogr2ogr

command line tool).

cmd = ["ogr2ogr", "-f", driver, output_file, self.conn_str, ] if sql_query: cmd.extend([sql_query_flag]) cmd.extend([source, "-nlt", "PROMOTE_TO_MULTI", "-nln", table_name if table_name else "query_result", ] )

When I run the flow manually from within a docker container running the image I'm using, it works fine, but when I deploy the flow and run it from a quick run in Prefect Cloud (which uses a work pool that creates a KubernetesJob in our cluster), it fails with

[Errno 2] No such file or directory: 'ogr2ogr'

It looks like your issue arises because the

ogr2ogr

command isn't found within the Kubernetes environment. Here are a few steps to troubleshoot and resolve this: 1. Ensure the Image Has All Dependencies: Verify that the Docker image used in your Kubernetes job has GDAL installed. You might need a custom Docker image. 2. Check Image Source in Your Deployment Configuration (prefect.yaml): Ensure your deployment YAML is correctly pointing to the custom Docker image that includes GDAL. Your configuration may look like this:

yaml
    version: 2
    flows:
      your-flow-name:
        parameters:
          ...
        schedule:
          ...
        storage:
          block: gcs/my-bucket
        run_configs:
          kubernetes:
            job:
              spec:
                ...
            image: "your-custom-image:latest"

3. Volume Mounts: If GDAL is installed on a mounted volume, make sure the volume is correctly mounted in your Kubernetes job. 4. Locales and PATH: Ensure the binaries are in the PATH for your subprocess call. You might need to adjust your Dockerfile to set up the environment properly. 5. Use Prefect Docker Infrastructure: Prefect may use a different base image internally. Here’s how you can specify the image correctly:

yaml
    infrastructure:
      type: kubernetes
      env: 
         ...
      image: "your-custom-image"

### Useful Links - Prefect Kubernetes Deployment Guide - Prefect Infrastructure Concept - Prefect Docker Image Examples By ensuring these steps are correctly configured, you should be able to run your subprocess without issues in a Kubernetes job.