Hi I am struggling with Kubernetes Config block copied the c Prefect Community #ask-community

Hi, I am struggling with Kubernetes Config block, ...

eddy davies

10/31/2022, 4:58 PM

Hi, I am struggling with Kubernetes Config block, copied the contents of

config

file in my

.kube

file but doesn't seem to work, any advice?

✅ 1

eddy davies

10/31/2022, 4:59 PM

Copy code

prefect deployment build log_flow.py:log_flow -n log-flow-s3 -sb s3/hubspot -q test -o log-flow-s3-k8s-deployment.yaml -t test -ib kubernetes-job/k8s-demo

This works without the

-ib

tag but obviously runs as process and expects me to start an agent locally

eddy davies

10/31/2022, 5:03 PM

I have an agent running using the helm chart with

helm install --values k8s-agent.yaml prefect-agent prefect/prefect-agent

where the

--values

yaml is

Copy code

agent:  
  image:
    repository: prefecthq/prefect
    prefectTag: 2-python3.10
    pullPolicy: IfNotPresent

  config:
    workQueues:
      - test

  cloudApiConfig:
    accountId: ""
    workspaceId: ""

with ids added

eddy davies

10/31/2022, 5:04 PM

prefect api key added as secret as well

kubectl apply -f prefect-secrets.yaml

Copy code

apiVersion: v1
stringData:
  key: ""
kind: Secret
metadata:
  name: prefect-api-key
  namespace: default
type: Opaque

Nate

10/31/2022, 5:13 PM

hey @eddy davies how are you attempting to use the

KubernetesClusterConfig

block? is it specified within your

kubernetes-job/k8s-demo

block? also, are you seeing some sort of error in your agent logs?

eddy davies

10/31/2022, 5:29 PM

Yes and then error seems to be an issue with config format, here is the end of it:

Copy code

block = block_cls.parse_obj(block_document.data)
  File "pydantic/main.py", line 521, in pydantic.main.BaseModel.parse_obj
  File "/Users/edwarddavies/.pyenv/versions/3.10.5/envs/prefect-k8s/lib/python3.10/site-packages/prefect/blocks/core.py", line 175, in __init__
    super().__init__(*args, **kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for KubernetesJob
cluster_config -> config
  value is not a valid dict (type=type_error.dict)
An exception occurred.

Nate

10/31/2022, 5:51 PM

hmm - could you try creating your

KubernetesClusterConfig

block using its

from_file

method? so like in ipython or something you could do

Copy code

from prefect.blocks.kubernetes import KubernetesClusterConfig

my_new_k8s_config = KubernetesClusterConfig.from_file(path="~/.kube/config")

my_new_k8s_config.save("my-new-k8s-config-block-name")

eddy davies

10/31/2022, 10:04 PM

That deployment is now building and applying correctly, so some issue with copying from config file into the ui

🦜 1

eddy davies

10/31/2022, 10:04 PM

thanks

👍 1

5 Views

Open in Slack

Previous Next