nicholasnet
02/07/2023, 4:05 PMredsquare
02/07/2023, 4:06 PMnicholasnet
02/07/2023, 4:10 PMredsquare
02/07/2023, 4:11 PMinfra_overrides = dict()
customizations = []
#create patch for each secret store
for secret in secret_stores:
secretPatch = {
"op": "add",
"path": "/spec/template/spec/containers/0/envFrom/-",
"value": {
"secretRef": {
"name": secret.lower()
}
}}
customizations.append(secretPatch)
infra_overrides['customizations'] = customizations
Deployment.build_from_flow(
...
infra_overrides = infra_overrides
)
nicholasnet
02/07/2023, 4:16 PMredsquare
02/07/2023, 4:19 PMnicholasnet
02/07/2023, 4:22 PMredsquare
02/07/2023, 4:23 PMnicholasnet
02/07/2023, 4:24 PMprefect deployments build \ flows/flow.py:kubernetes_flow \
--name prefect-kubernetes-example \
--infra-block kubernetes-job/prefect-kubernetes-poc \
--work-queue kubernetes \
--apply
redsquare
02/07/2023, 4:24 PM--override
nicholasnet
02/07/2023, 4:25 PMredsquare
02/07/2023, 4:26 PMnicholasnet
02/07/2023, 4:30 PMredsquare
02/07/2023, 4:33 PMnicholasnet
02/07/2023, 4:39 PMyaml.composer.ComposerError: expected a single document in the stream
in "common.yml", line 1, column 1
but found another document
in "common.yml", line 19, column 1
This is my yaml
apiVersion: batch/v1
kind: Job
metadata:
# labels are required, even if empty
labels: { }
spec:
template:
spec:
completions: 1
containers: # the first container is required
- env: [ ] # env is required, even if empty
name: prefect-job
envFrom:
- secretRef:
name: "bidw-access-control-service"
parallelism: 1
restartPolicy: Never
---
apiVersion: "<http://koudingspawn.de/v1|koudingspawn.de/v1>"
kind: Vault
metadata:
name: bidw-access-control-service
namespace: dev-bidw
spec:
path: "dev/bidw-access-control-service"
type: "KEYVALUEV2"
and this is my block.py
from prefect.infrastructure.kubernetes import KubernetesJob, KubernetesImagePullPolicy
def create_kubernetes_job():
block = KubernetesJob(
image="someimage",
namespace="dev-bidw",
image_pull_policy=KubernetesImagePullPolicy.ALWAYS,
finished_job_ttl=120,
job_watch_timeout_seconds=6000,
pod_watch_timeout_seconds=6000,
env={"environment": "dev"},
job=KubernetesJob.job_from_file("common.yml")
)
block.save("prefect-kubernetes-job-poc", overwrite=True)
if __name__ == "__main__":
create_kubernetes_job()
redsquare
02/21/2023, 11:43 PMnicholasnet
02/21/2023, 11:44 PMredsquare
02/21/2023, 11:47 PMnicholasnet
02/21/2023, 11:51 PMredsquare
02/21/2023, 11:57 PMnicholasnet
02/21/2023, 11:57 PMredsquare
02/21/2023, 11:58 PMnicholasnet
02/22/2023, 12:00 AMredsquare
02/22/2023, 12:07 AMnicholasnet
02/22/2023, 3:38 AMredsquare
02/22/2023, 6:12 AM