https://prefect.io logo
Join Slack
Powered by
Good day! I'm trying to convert my Prefect 1 to Pr...
# prefect-kubernetes
b
Good day! I'm trying to convert my Prefect 1 to Prefect 2. In 1 I was able to run an agent pod in kubernetes and just assign it a job template yaml file to use to run flows. I'd like to do something similar in 2. I've read a little bit about infrastructure blocks, but ideally I just want to define it with the same job yaml file. Is this possible to do in Prefect 2 without an infrastructure block?
✅ 1
k
rather than an agent, you can use a kubernetes work pool with an attached worker that has a base job definition instead of creating infra blocks for all your deployments
🙌 1
b
I've been playing around with that too. So I'd define a worker. Then would be able to give it a job template yaml that includes an image I want it to use and env vars that link to kubernetes secrets?
k
when you create the work pool there will be a job template you can fill out, very similar to the kubernetes job block, where you can define things like the image and secrets and so on
👍 1
upvote 1
🙏 1
b
So, quick follow up question: Is it possible to point that to the kubernetes secretKeyRef ex:
{
"PYODBC_CONN_STR": {
"valueFrom": {
"secretKeyRef": {
"name": "prefect-env",
"key": "PYODBC_CONN_STR"
}
}
}
}
k
when you're creating your kubernetes work pool in the prefect UI, switch the base job template to 
advanced
and then you can edit it like a typical kubernetes job manifest, including for your env vars
👍 2
you should also be able to place that part in the environment variables portion of the default template
b
Update: That json does not work, it expects the format to be
Copy code
{
  "key": "value",
  "ENVIRONMENT": "dev"
}
Even with that, passing in the right side ends up just being a string, and does not load from the kubernetes secrets. Or fails completely if you attempt to pass an object.
k
this is if you try to fill out the env var section in the ui?
b
Correct.
k
what if you tried adding that json to the advanced job template for the work pool?
b
I'm seeing a couple of places where there are 
{{ env }}
variables in that template. Could it be under the 
job_manifest/spec/template/spec/containers/{[env]}
?
Copy code
"job_manifest": {
      "kind": "Job",
      "spec": {
        "template": {
          "spec": {
            "containers": [
              {
       ------>> "env": "{{ env }}",
                "args": "{{ command }}",
                "name": "prefect-job",
                "image": "{{ image }}",
                "imagePullPolicy": "{{ image_pull_policy }}"
              }
            ],
k
I think it may have been the formatting of your json. go back to the default work pool template and try this as your env var
Copy code
{
  "name": "PYODBC_CONN_STR",
  "valueFrom": {
      "secretKeyRef": {
        "key": "PYODBC_CONN_STR",
        "name": "prefect-env"
      }
    }
  }
}
ahhhh okay I'm looking at the worker code
Copy code
self.job_manifest["spec"]["template"]["spec"]["containers"][0]["env"] = [
     {"name": k, "value": v} for k, v in self.env.items()
 ]
what you said is correct. 
env
is expecting just 
"name": "value"
pairs.
👍 1
it seems like 
env
is inherited from 
prefect.workers.base.BaseJobConfiguration
, which defines 
env
like:
Copy code
class BaseJobConfiguration(BaseModel):
    ...
    env: Dict[str, Optional[str]] = Field(
        default_factory=dict,
        title="Environment Variables",
        description="Environment variables to set when starting a flow run.",
    )
I would try replacing the 
{{ env }}
part of the template, but given how this is written, I don't think that would work
b
So it'll probably work the same way as the UI. it expects a string and only a string, and will fail if an object is used. Thanks for looking at that @Kevin Grismore! I'm going to just go with secret blocks. I was trying to avoid them so I didn't have another place I'd have to configure things!
Open in Slack
PreviousNext
Powered by