https://prefect.io
Join Slack
<@ULVA73B9P> is this correc structur of basejobtemplate in values.yamo file "baseJobTemplate: ...
# marvin-ai
v
@Marvin is this correc structur of basejobtemplate in values.yamo file "baseJobTemplate: configuration: |+ { "apiVersion": "batch/v1", "kind": "Job", "metadata": { "labels": { "app": "prefect-worker" } }, "spec": { "template": { "spec": { "containers": [ { "name": "prefect-worker", "image": "mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7", "env": [ { "name": "POSTGRES_DBNAME", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "POSTGRES_DBNAME" } } }, { "name": "POSTGRES_HOST", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "POSTGRES_HOST" } } }, { "name": "POSTGRES_USER", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "POSTGRES_USER" } } }, { "name": "POSTGRES_PORT", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "POSTGRES_PORT" } } }, { "name": "DBT_SCHEMA", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "DBT_SCHEMA" } } }, { "name": "DBT_THREADS", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "DBT_THREADS" } } }, { "name": "DISTRIBUTION_ELT_SERVICE_URL", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "DISTRIBUTION_ELT_SERVICE_URL" } } }, { "name": "ENV", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "ENV" } } }, { "name": "DISTRIBUTION_DEPLOYMENT_ID", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "DISTRIBUTION_DEPLOYMENT_ID" } } }, { "name": "PF_API_URL", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "PF_API_URL" } } }, { "name": "ELT_URL", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "ELT_URL" } } }, { "name": "RP_POSTGRES_USER", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "RP_POSTGRES_USER" } } }, { "name": "SALESFORCE_DOMAIN", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "SALESFORCE_DOMAIN" } } }, { "name": "NOTIFY_SENDER_EMAIL", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "NOTIFY_SENDER_EMAIL" } } }, { "name": "SALESFORCE_USERNAME", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "SALESFORCE_USERNAME" } } }, { "name": "PREFECT_LOGGING_LEVEL", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "PREFECT_LOGGING_LEVEL" } } }, { "name": "NOTIFY_RECIPIENT_EMAIL", "valueFrom": { "configMapKeyRef": { "name": "prefect-worker-configmap", "key": "NOTIFY_RECIPIENT_EMAIL" } } }, { "name": "POSTGRES_PASSWORD", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "POSTGRES_PASSWORD" } } }, { "name": "ENCRYPTION_SECRET_KEY", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "ENCRYPTION_SECRET_KEY" } } }, { "name": "ENCRYPTION_IV", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "ENCRYPTION_IV" } } }, { "name": "SENDGRID_API_KEY", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "SENDGRID_API_KEY" } } }, { "name": "IMPRESSIONWISE_PWD", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "IMPRESSIONWISE_PWD" } } }, { "name": "IMPRESSIONWISE_CODE", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "IMPRESSIONWISE_CODE" } } }, { "name": "SALESFORCE_PASSWORD", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "SALESFORCE_PASSWORD" } } }, { "name": "RP_POSTGRES_PASSWORD", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "RP_POSTGRES_PASSWORD" } } }, { "name": "SALESFORCE_SECURITY_TOKEN", "valueFrom": { "secretKeyRef": { "name": "dev-worker-secrets", "key": "SALESFORCE_SECURITY_TOKEN" } } } ] } ], "restartPolicy": "Never" } } } }"
m
✅ thought for 30.7 seconds
Your 
baseJobTemplate
structure is generally correct, but there are a few things to note and potential improvements: 1. The image URL in your configuration contains an invalid format:
Copy code
yaml
"image": "<http://mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7|mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7>"
Should be simply:
Copy code
yaml
"image": "<http://mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7|mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7>"
2. You might want to consider adding resource limits and requests:
Copy code
yaml
"resources": {
  "requests": {
    "cpu": "100m",
    "memory": "128Mi"
  },
  "limits": {
    "cpu": "500m",
    "memory": "512Mi"
  }
}
3. Consider adding 
imagePullPolicy
if working with private registries:
Copy code
yaml
"imagePullPolicy": "Always"
4. The overall structure with your environment variables using 
configMapKeyRef
and 
secretKeyRef
is correct and follows best practices for Kubernetes secret management. Here's a cleaner way to format the YAML:
Copy code
yaml
baseJobTemplate:
  configuration: |
    {
      "apiVersion": "batch/v1",
      "kind": "Job",
      "metadata": {
        "labels": {
          "app": "prefect-worker"
        }
      },
      "spec": {
        "template": {
          "spec": {
            "containers": [
              {
                "name": "prefect-worker",
                "image": "<http://mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7|mmcdevaksreg.azurecr.io/mmc_updb_data_marts:20240712.7>",
                "imagePullPolicy": "Always",
                "env": [
                  # Your existing env vars are correct
                ]
              }
            ],
            "restartPolicy": "Never"
          }
        }
      }
    }
The structure you're using is valid for a Kubernetes work pool configuration, but make sure to: 1. Fix the image URL format 2. Consider adding resource limits/requests 3. Add 
imagePullPolicy
if needed 4. Ensure proper indentation in your YAML file These configurations will be used as a template for all jobs spawned by this work pool, so it's important to get the structure right.
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by