https://prefect.io logo
Join Slack
Powered by
Hey, guys. How you doing? I've exported fully con...
# prefect-kubernetes
f
Hey, guys. How you doing? I've exported fully config of my kubernetes cluster using this command: 
$ kubectl config view --output json --raw=false --minify --flatten > cluster.json
and created a block of Kubernetes Cluster Config on the UI. All running good, but when i'm trying to load the block, i got and 404 and it says dont have a block called xxxyyy. The work-poll was created and the block config too, as you can see. This is a simple test to creating deployments on kube and running in the work-pools.
Copy code
from prefect import flow
from prefect.blocks.kubernetes import KubernetesClusterConfig
cluster_config_block = KubernetesClusterConfig.load("winker-netes")
from prefect.deployments import DeploymentImage


@flow(log_prints=True)
def hello_world(name: str = "world", goodbye: bool = False):
    print(f"Hello {name} from Prefect! 🤗")

    if goodbye:
        print(f"Goodbye {name}!")


if __name__ == "__main__":
    hello_world.deploy(name="fabio-deployment",
                tags=["onboarding"],
                parameters={"goodbye": True},
                interval=60,
                work_pool_name="winkernetes-bigdata",
                image=DeploymentImage(
                    name="tiwinker/dwh:hello-world",
                    platform="linux/amd64"
    ))
So, what iḿ doing wrong here? In time: iḿ using prefect community. Thanks for any help.
n
hi @Fabio Machado - this seems like it could be a bug can you share the versions of prefect / prefect-kubernetes you are using?
f
for sure: prefect-server: 
2.20.2
prefect-kubernetes: prefect-kubernetes==0.4.1
in time: its possible to use taints and tolerations in prefect.yaml ou other place? I've separated worker nodes to hosting prefect and prefect jobs.
n
hi @Fabio Machado - i've opened a PR to fix this https://github.com/PrefectHQ/prefect/pull/15095 ill get a release out for this asap and then come back to your question about k8s config
f
thank you @Nate
n
the release is out! please let me know if you hit any other issues
f
thanks, Nate. I'll be back again if i find errors
🙏 1
Hey @Nate. How you doing. My friend, until i waiting for your PR being approved, i've changed the approach and wroted one prefect.yaml file to deploy our test code on kubernetes. Probably i've made a mistake, because the image that i'm builded give to me an error about path. Below my prefect.yaml. The kubernetes block on this method was working very good, but when the pods is running, i got this error below.
Copy code
python: can't open file '/code/prefect': [Errno 2] No such file or directory
Dockerfile to build the image:
Copy code
FROM prefecthq/prefect:2-python3.12

RUN apt update && \
apt install -y coreutils

WORKDIR /code

COPY hello_world.py .

ENTRYPOINT [ "python" ]
CMD [ "hello_world.py" ]
Copy code
version: '2.0'
name: fabio-deploy-kube28

description: Deployment Kubernetes

work_pool:
  name: winkernetes-27  

infrastructure:
  type: kubernetes-job
  blockName: winker-netes  
  working_directory: {}
  image: myrepo/dwh:hello-world
  tolerations:
    - key: "name"
      operator: "Equal"
      value: "prefect"
      effect: "NoSchedule"
      tolerationSeconds: 3600  
  resources:
    limits:
      cpu: "1"
      memory: "512Mi"
    requests:
      cpu: "0.5"
      memory: "256Mi"


# Aqui você pode definir variáveis de ambiente que serão utilizadas no job Kubernetes
parameters:
  nome: fabio
  sobrenome: machado

tags:
- kubernetes
- meu-fluxo

schedule:
  cron: '*/1 * * * *'  

# storage:
  # type: remote-file-system
  # blockName: "meu-storage-block"  # Substitua pelo nome do bloco de armazenamento remoto, se necessário

deployments:
- name: teste-fabio-27
  entrypoint: hello_world.py:hello_world
  version: '1.0'
  tags:
  - kubernetes
  - meu-fluxo
  parameters:
    nome: fabio
    sobrenome: machado
  description:
  work_pool:
    name: winkernetes-27
    work_queue_name: default
    job_variables:
        env:
            PREFECT_API_URL: <http://prefect-server.big-data.svc.cluster.local:4200/api>
        image: '{{ build-image.image }}'
  schedules:
  - cron: '*/1 * * * *'
    timezone:
    day_or: true
    active: true
  build:
  - prefect_docker.deployments.steps.build_docker_image:
      requires: prefect-docker>=0.3.1
      id: build-image
      dockerfile: ./Dockerfile
      image_name: myrepo/dwh
      tag: hello-world
      dockerfile: ./Dockerfile      
  push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker>=0.3.1
      image_name: '{{ build-image.image_name }}'
      tag: '{{ build-image.tag }}'
      credentials: '{{ prefect_docker.docker-registry-credentials.docker_registry_creds_name
        }}'
  pull:
  - prefect.deployments.steps.set_working_directory:
      directory: {}
4 Views
Open in Slack
PreviousNext
Powered by