https://prefect.io logo
Title
n

nicholasnet

02/07/2023, 4:05 PM
How do you attach secret block in Kubernetes Job and how it can access those secrets during runtime?
r

redsquare

02/07/2023, 4:06 PM
you have to do it as a patch document and add it to the deployment infra_overrides
n

nicholasnet

02/07/2023, 4:10 PM
So, you mean after deployment file is created we need to manually edit it and deploy it?
r

redsquare

02/07/2023, 4:11 PM
no - you can pass the patch into the build_from_flow
infra_overrides = dict()
customizations = []
#create patch for each secret store
for secret in secret_stores:
secretPatch = {
"op": "add",
"path": "/spec/template/spec/containers/0/envFrom/-",
"value": {
"secretRef": {
"name": secret.lower()
}
}}
customizations.append(secretPatch)
infra_overrides['customizations'] = customizations
Deployment.build_from_flow(
...
infra_overrides = infra_overrides
)
that is obviously passing all the secrets (secretRef) you can patch in a single one just the same
n

nicholasnet

02/07/2023, 4:16 PM
I am sorry I am new to all this. Where you put this patch.
r

redsquare

02/07/2023, 4:19 PM
See the build_from_flow
Your passing in a dictionary of infra_overrides (json patches for the job template)
n

nicholasnet

02/07/2023, 4:22 PM
Thank you was browsing same document 🙂 . Let me review it.
r

redsquare

02/07/2023, 4:23 PM
your just passing in a list of jsonpatches
n

nicholasnet

02/07/2023, 4:24 PM
But right now I am using command like this to deploy.
prefect deployments build \                                                                                          flows/flow.py:kubernetes_flow \
    --name prefect-kubernetes-example \
    --infra-block kubernetes-job/prefect-kubernetes-poc \
     --work-queue kubernetes \
                --apply
r

redsquare

02/07/2023, 4:24 PM
yeah I use python to deploy
command line has
--override
n

nicholasnet

02/07/2023, 4:25 PM
How you can deploy using plain python I thought you can do so only using that command.
r

redsquare

02/07/2023, 4:26 PM
prefect has a python api to do most things
I find it easier to template up deployments from flows using cookiecutter - easier to debug too
n

nicholasnet

02/07/2023, 4:30 PM
Is there any documentation or video somewhere. I only found command line way of doing it.
r

redsquare

02/07/2023, 4:33 PM
n

nicholasnet

02/07/2023, 4:39 PM
Thank you very much.
👍 1
It looks like multi document is not supported Getting this error
yaml.composer.ComposerError: expected a single document in the stream
  in "common.yml", line 1, column 1
but found another document
  in "common.yml", line 19, column 1
This is my yaml
apiVersion: batch/v1
kind: Job
metadata:
    # labels are required, even if empty
    labels: { }
spec:
    template:
        spec:
            completions: 1
            containers: # the first container is required
                -   env: [ ]  # env is required, even if empty
                    name: prefect-job
                    envFrom:
                        -   secretRef:
                                name: "bidw-access-control-service"
            parallelism: 1
            restartPolicy: Never

---
apiVersion: "<http://koudingspawn.de/v1|koudingspawn.de/v1>"
kind: Vault
metadata:
    name: bidw-access-control-service
    namespace: dev-bidw
spec:
    path: "dev/bidw-access-control-service"
    type: "KEYVALUEV2"
and this is my block.py
from prefect.infrastructure.kubernetes import KubernetesJob, KubernetesImagePullPolicy


def create_kubernetes_job():
    block = KubernetesJob(
        image="someimage",
        namespace="dev-bidw",
        image_pull_policy=KubernetesImagePullPolicy.ALWAYS,
        finished_job_ttl=120,
        job_watch_timeout_seconds=6000,
        pod_watch_timeout_seconds=6000,
        env={"environment": "dev"},
        job=KubernetesJob.job_from_file("common.yml")
    )
    block.save("prefect-kubernetes-job-poc", overwrite=True)


if __name__ == "__main__":
    create_kubernetes_job()
Is there anyway to add this?
r

redsquare

02/21/2023, 11:43 PM
do you not just use annotations in the pod to get the secrets?
n

nicholasnet

02/21/2023, 11:44 PM
No I am using Vault to pull secret out. I was able to do this when deploying agent though.
But I had to manually edit yaml
r

redsquare

02/21/2023, 11:47 PM
4f9e5d2f-57b0-40d0-bc60-5650008cfef3.jpeg
isnt it like that? I use rancher, never touched hashicorp stuff
n

nicholasnet

02/21/2023, 11:51 PM
No we use this CRD https://vault.koudingspawn.de/ for Vault. Since it restarts pods if secrets get changed at Vault side.
r

redsquare

02/21/2023, 11:57 PM
so your trying to create the secret from vault into k8s
as part of this deployment
n

nicholasnet

02/21/2023, 11:57 PM
Yeah
r

redsquare

02/21/2023, 11:58 PM
yeah, that wont work
n

nicholasnet

02/22/2023, 12:00 AM
Oh 😬 . Let me see if I can make it work other way.
r

redsquare

02/22/2023, 12:07 AM
n

nicholasnet

02/22/2023, 3:38 AM
Looks like I found way to achieve this without multi doc support. Thank you very much for your help.
r

redsquare

02/22/2023, 6:12 AM
cool