Hi everyone, I am hoping someone can help me out with an issue related to overriding the image of a deployment for a specific run. For the past few hours, I have been struggling trying to override the

image

url of my work pool (GCP Cloud Run Push) using the Create Flow Run From Deployment API Call. This API call has a body item called

jobs_variables

. The body looks as follows:

{
  "state": {
    "type": "SCHEDULED"
  },
  "job_variables": {
    "image": "eu.gcr.io/abc/prefect/prefect-pricing:development"
  }
}

After the API is called and the workflow starts, the flow does not run using the specified image URL, instead it falls back to the default value. Yet, I do see the value in the Flow's job variables. My work pool's job template looks as follows:

{
  "variables": {
    "type": "object",
    "required": [
      "credentials"
    ],
    "properties": {
      "image": {
        "type": "string",
        "title": "Image Name",
        "default": "eu.gcr.io/xxx/prefect/prefect-pricing:production",
        "description": "The image to use for the Cloud Run job. If not provided the latest Prefect image will be used."
      },
...
"job_configuration": {
    "env": "{{ env }}",
    "name": "{{ name }}",
    "labels": "{{ labels }}",
    "region": "{{ region }}",
    "command": "{{ command }}",
    "timeout": "{{ timeout }}",
    "job_body": {
      "client": "prefect",
      "template": {
        "template": {
          "timeout": "{{ timeout }}",
          "containers": [
            {
              "env": [],
              "args": "{{ args }}",
              "image": "{{ image }}",
              "command": "{{ command }}",
              "resources": {
                "limits": {
                  "cpu": "{{ cpu }}",
                  "memory": "{{ memory }}"
                }
              }
            }
...
  }
}

Why is the image value not overriding the default value? What am I doing wrong/missing? Thanks 😄

you can schedule support time here for time-sensitive problems, there's lot of questions in community and I do my best to get to them as time allows. not tagging is appreciated are you unable to use the sdk in this case?

I would ideally use the API. Yet I am facing the same problem with the CLI. Overriding the

image

variable in a work pool works when I hardcode the

image

value in a deployment, as job variable. Yet, when I try to set the image job variable in a deployment as a variable like:

{
  "image": "{{ $gcr_image }}"
}

and then pass it in my CLI command like

prefect deployment run \
--id "de27fd5c-eab1-4f40-8bbc-7a6684e1ceb2" \
--job-variable gcr_image="eu.gcr.io/abc/prefect/prefect-pricing:production"

the flow run will yield this error.

Expected an image path like [host/]repo-path[:tag and/or @digest], where host is one of [region.]gcr.io, [region-]docker.pkg.dev or docker.io but obtained {{ $gcr_image }}'}]}]">

what kind of variable are you trying to set here?

{
  "image": "{{ $gcr_image }}"
}

that is the syntax for when you want it to render an env var set on your local named

gcr_image

if you want a prefect variable value to be rendered, you should use

{
  "image": "{{ prefect.variables.gcr_image }}"
}

Thanks for the help Nate, I’m working with a single environment, therefore no Prefect Variables are used. Similarly, I do not want to declare a image at deployment of a flow. Instead, I have set the work pool to a default image value (production image). I expect Prefect to override this value whenever a deployment flow run is triggered with

-jv syntax

included.

I expect Prefect to override this value whenever a deployment flow run is triggered with

-jv syntax

included.

this is how it should work today is the override failing specifically when you try from the UI?

The override fails for all options I tried, UI, CLI, and API.

afaict yes. we go over this here if you're still running into trouble, can you be more exact about what you tried / how you tried it?

Thanks, Nate. I have also reviewed the documentation you're referring to. Unfortunately, I did not fully grasp why the

env

variable was added as a parameter in the screenshot on this page. I've detailed the steps I undertook below. Hopefully, this will help you understand the issue and reproduce it. Work Pool: The work pool I use is a Cloud Run Push V2. No base image is defined. Hence, if no image is passed in the deployment or flow run, it defaults back to the Prefect base image. Deployment: My flow is deployed using the

.deploy

method:

if __name__ == "__main__":
    process_pricing_xx_files.deploy(
        parameters={"host": '<http://sysmgr.xxx.com|sysmgr.xxx.com>', "username": 'FTP-Agent'},
        work_pool_name="cloud-run-push-pricing",
        image="<http://eu.gcr.io/xxx/prefect/prefect-pricing:production|eu.gcr.io/xxx/prefect/prefect-pricing:production>",
        build=False
    )

Once deployed, I can see on the deployment page in the Prefect Cloud UI that the image is correctly passed as a job variable under the

configuration

tab.

{
  "image": "<http://eu.gcr.io/xxx/prefect/prefect-pricing:production|eu.gcr.io/xxx/prefect/prefect-pricing:production>"
}

When triggered, using the

quick run

option, Google Cloud Run pulls the image as defined in the deployment. So far so good. Custom Flow Run This is where the unexpected behavior occurs. Whenever I perform a custom run of this deployment by triggering a flow run, I am unable to override the

image

value. I have tried multiple methods. Below, I describe the CLI approach: I execute the following command in the CLI (note that I am specifying

development

as the image tag, rather than the default

production

):

prefect deployment run \
--id "de27fd5c-eab1-4f40-8bbc-7a6684e1ceb2" \
--job-variable image="<http://eu.gcr.io/xxx/prefect/prefect-pricing:development|eu.gcr.io/xxx/prefect/prefect-pricing:development>"

Prefect creates a flow run and correctly reads back my variables, noting the flow run name, dramatic-swan:

Creating flow run for deployment 'process-pricing-xx-files/1tis_supplier_pricing_xx'...
Created flow run 'dramatic-swan'.
└── UUID: 8eb04784-b910-43b9-8e94-48c91fc93ac1
└── Parameters: {'host': '<http://sysmgr.xx.com|sysmgr.xx.com>', 'username': 'FTP-Agent'}
└── Job Variables: {'image': '<http://eu.gcr.io/xxx/prefect/prefect-pricing:development|eu.gcr.io/xxx/prefect/prefect-pricing:development>'}
└── Scheduled start time: 2024-06-21 09:54:09 CEST (now)
└── URL: <https://app.prefect.cloud/account/xx/workspace/xx/flow-runs/flow-run/8eb04784-b910-43b9-8e94-48c91fc93ac1>

I can also see on the flow run page in the UI that under Job Variables, the image is overridden as expected:

{
  "image": "<http://eu.gcr.io/xxx/prefect/prefect-pricing:development|eu.gcr.io/xxx/prefect/prefect-pricing:development>"
}

Yet, in Google Cloud Run, as the attached image shows, it did not override the image URL with the

development

tag; instead it used

production

, the default value. In conclusion • Prefect passes the image variable set during deployment correctly and normal quick runs work as expected. • Custom runs with job variables that should override the image variable of a deployment are not actually overridden Please let me know if you need additional details and thanks again!

thank you for the detail! i will look into this when I can

Hi @Nate , Have you had a chance to look at the above? Thanks again!

hi @Martijn van Elferen - i have not been able to reproduce your issue so far can you open an issue with the important detail from your setup if there is not one on this?