https://prefect.io logo
#prefect-community
Title
# prefect-community
s

Shivam Bhatia

07/14/2022, 7:46 AM
Hi, I just hosted a prefect 1.2.4 on a gcp vm I am trying to run a flow using VertexRun and I got this error
Copy code
400 List of found errors:	1.Field: job_spec.worker_pool_specs[0].container_spec.env[12].value; Message: Required field is not set.	2.Field: job_spec.worker_pool_specs[0].container_spec.env[3].value; Message: Required field is not set.	3.Field: job_spec.worker_pool_specs[0].container_spec.env[4].value; Message: Required field is not set.	 [field_violations {
  field: "job_spec.worker_pool_specs[0].container_spec.env[12].value"
  description: "Required field is not set."
}
field_violations {
  field: "job_spec.worker_pool_specs[0].container_spec.env[3].value"
  description: "Required field is not set."
}
field_violations {
  field: "job_spec.worker_pool_specs[0].container_spec.env[4].value"
  description: "Required field is not set."
}
]
Do i need to configure vertex jobs manually?
1
a

Anna Geller

07/14/2022, 10:38 AM
No, you don't have to. It can be set when starting the agent: https://docs.prefect.io/orchestration/agents/vertex.html
s

Shivam Bhatia

07/14/2022, 5:22 PM
So basically I used the same setup on prefect cloud Running a docker container on the vertex agent Did the same thing on a self hosted instance but I'm getting this error Is this a gcp error or a prefect error?
a

Anna Geller

07/14/2022, 6:32 PM
it looks like you would either need to add more info during the agent start command or provide more info to the VertexRun run config to fill those missing required fields
s

Shivam Bhatia

07/15/2022, 8:39 AM
I looked at a vertex job config for an older flow that i had created using the prefect cloud and i found this config
Copy code
{
  "workerPoolSpecs": [
    {
      "machineSpec": {
        "machineType": "e2-highmem-8"
      },
      "replicaCount": "1",
      "diskSpec": {
        "bootDiskType": "pd-ssd",
        "bootDiskSizeGb": 100
      },
      "containerSpec": {
        "imageUri": "<image_url> (on dockerhub)",
        "command": [
          "prefect",
          "execute",
          "flow-run"
        ],
        "env": [
          {
            "name": "PREFECT__LOGGING__LEVEL",
            "value": "INFO"
          },
          {
            "name": "PREFECT__BACKEND",
            "value": "cloud"
          },
          {
            "name": "PREFECT__CLOUD__API",
            "value": "<https://api.prefect.io>"
          },
          {
            "name": "PREFECT__CLOUD__API_KEY",
            "value": "<api key>"
          },
          {
            "name": "PREFECT__CLOUD__TENANT_ID",
            "value": "<tenant id>"
          },
          {
            "name": "PREFECT__CLOUD__AGENT__LABELS",
            "value": "['dev']"
          },
          {
            "name": "PREFECT__CLOUD__SEND_FLOW_RUN_LOGS",
            "value": "true"
          },
          {
            "name": "PREFECT__CONTEXT__FLOW_RUN_ID",
            "value": "<flow id>"
          },
          {
            "name": "PREFECT__CONTEXT__FLOW_ID",
            "value": "<flow id>"
          },
          {
            "name": "PREFECT__CLOUD__USE_LOCAL_SECRETS",
            "value": "false"
          },
          {
            "name": "PREFECT__ENGINE__FLOW_RUNNER__DEFAULT_CLASS",
            "value": "prefect.engine.cloud.CloudFlowRunner"
          },
          {
            "name": "PREFECT__ENGINE__TASK_RUNNER__DEFAULT_CLASS",
            "value": "prefect.engine.cloud.CloudTaskRunner"
          },
          {
            "name": "PREFECT__LOGGING__LOG_TO_CLOUD",
            "value": "true"
          },
          {
            "name": "PREFECT__CLOUD__AUTH_TOKEN",
            "value": "<prefect auth key>"
          }
        ]
      }
    }
  ],
  "serviceAccount": "<service_account>"
}
The initial error says
Copy code
[field_violations {
  field: "job_spec.worker_pool_specs[0].container_spec.env[12].value"
  description: "Required field is not set."
}
field_violations {
  field: "job_spec.worker_pool_specs[0].container_spec.env[3].value"
  description: "Required field is not set."
}
field_violations {
  field: "job_spec.worker_pool_specs[0].container_spec.env[4].value"
  description: "Required field is not set."
}
]
env[3] and env[4] are cloud api key and tenant id env[12] is PREFECT_LOGGING_LOG_TO_CLOUD Since i am not using prefect cloud and i have a self hosted instance of prefect 1.2.4 How do i workaround the required fields of cloud api key?
a

Anna Geller

07/15/2022, 4:49 PM
the env variables should look like this (you'd need to adapt it to your JSON spec):
Copy code
env:
        - name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
          value: ''
        - name: PREFECT__CLOUD__API
          value: "<http://some_ip:4200/graphql>" # paste your GraphQL Server endpoint here
        - name: PREFECT__BACKEND
          value: server
Copy code
PREFECT__CLOUD__API_KEY
can also be set to an empty string
s

Shivam Bhatia

07/16/2022, 5:58 AM
I have updated the config to this:
Copy code
worker_pool_specs = [
        {
            "machine_spec": {"machine_type": "e2-highmem-8"},
            "replica_count": 1,
            "container_spec": {
                "imageUri": "<url to my docker image>",
                "command": [
                    "prefect",
                    "execute",
                    "flow-run"
                ],
                "env": [
                    {
                        "name": "PREFECT__LOGGING__LEVEL",
                        "value": "INFO"
                    },
                    {
                        "name": "PREFECT__BACKEND",
                        "value": "server"
                    },
                    {
                        "name": "PREFECT__CLOUD__API",
                        "value": "http://<external ip to my vm>:4200/graphql"
                    },
                    {
                        "name": "PREFECT__CLOUD__API_KEY",
                        "value": ""
                    },
                    {
                        "name": "PREFECT__CLOUD__TENANT_ID",
                        "value": ""
                    },
                    {
                        "name": "PREFECT__CLOUD__AGENT__LABELS",
                        "value": "['dev']"
                    },
                    {
                        "name": "PREFECT__CLOUD__SEND_FLOW_RUN_LOGS",
                        "value": "true"
                    },
                    {
                        "name": "PREFECT__CONTEXT__FLOW_RUN_ID",
                        "value": ""
                    },
                    {
                        "name": "PREFECT__CONTEXT__FLOW_ID",
                        "value": ""
                    },
                    {
                        "name": "PREFECT__CLOUD__USE_LOCAL_SECRETS",
                        "value": "false"
                    },
                    {
                        "name": "PREFECT__ENGINE__FLOW_RUNNER__DEFAULT_CLASS",
                        "value": "prefect.engine.cloud.CloudFlowRunner"
                    },
                    {
                        "name": "PREFECT__ENGINE__TASK_RUNNER__DEFAULT_CLASS",
                        "value": "prefect.engine.cloud.CloudTaskRunner"
                    },
                    {
                        "name": "PREFECT__LOGGING__LOG_TO_CLOUD",
                        "value": ""
                    },
                    {
                        "name": "PREFECT__CLOUD__AUTH_TOKEN",
                        "value": ""
                    }
                ]
            },
        },
    ]
Still getting the same error
a

Anna Geller

07/16/2022, 12:27 PM
not sure why, perhaps for your PoC you can try the same with Prefect Cloud first only to see if this is working and then move to Server orchestration layer once you're sure the execution layer is working?
s

Shivam Bhatia

07/16/2022, 1:55 PM
I have tried the same flow with the same setup on prefect cloud before Everything's the same except In the prefect cloud execution i just passed the machine type and the docker image url as the parameters into the vertex run config
a

Anna Geller

07/16/2022, 4:41 PM
so it works with Cloud but not on Server? if so, could you open an issue in the server repo and describe your issue in detail there?
5 Views