Thread
#prefect-community
    Kayvan Shah

    Kayvan Shah

    4 months ago
    I observed that the pods start and keep failing and restarting While no resources show up when we get pods for default namespace
    Anna Geller

    Anna Geller

    4 months ago
    hard to determine the root cause just from those logs - are you running everything in a single K8s cluster? What storage do you use? could be your run can't pull the image or flow code
    restarting your agent process may be a simple step worth trying
    Kayvan Shah

    Kayvan Shah

    4 months ago
    Running on a single k8s cluster
    Also tried deleting everyting and restarting the minikube cluster but that didn't work as well
    Anna Geller

    Anna Geller

    4 months ago
    what did you do so far?
    this is for the OSS version - single deployment with all components https://orion-docs.prefect.io/tutorials/kubernetes-flow-runner/
    Kayvan Shah

    Kayvan Shah

    4 months ago
    Following the same doc
    can this happen when docker or minikube falls short of memory
    Anna Geller

    Anna Geller

    4 months ago
    What storage do you use? could be your run can't pull the image or flow code
    yup for sure it can, good call
    Kayvan Shah

    Kayvan Shah

    4 months ago
    File Storage
    Anna Geller

    Anna Geller

    4 months ago
    file storage won't work with K8s afaik
    you need remote storage such as S3, GCS, ABS
    K8s cluster can't access your local files and mounting is hard/too much complexity when compared to e.g. just S3
    Kayvan Shah

    Kayvan Shah

    4 months ago
    will try with S3 Hope that works
    I am trying to deploy on an on-premise server earlier when I had tried local storage creation of deployments failed but were created with file storage so I thought that things might work
    Anna Geller

    Anna Geller

    4 months ago
    gotcha, makes sense
    Kayvan Shah

    Kayvan Shah

    4 months ago
    ~/projects/prefect-demo$ minikube start
    šŸ˜„  minikube v1.25.2 on Ubuntu 18.04 (kvm/amd64)
    āœØ  Automatically selected the docker driver
    šŸ‘  Starting control plane node minikube in cluster minikube
    šŸšœ  Pulling base image ...
    šŸ”„  Creating docker container (CPUs=2, Memory=3944MB) ...
    šŸ³  Preparing Kubernetes v1.23.3 on Docker 20.10.12 ...
        ā–Ŗ kubelet.housekeeping-interval=5m
        ā–Ŗ Generating certificates and keys ...
        ā–Ŗ Booting up control plane ...
        ā–Ŗ Configuring RBAC rules ...
    šŸ”Ž  Verifying Kubernetes components...
        ā–Ŗ Using image <http://gcr.io/k8s-minikube/storage-provisioner:v5|gcr.io/k8s-minikube/storage-provisioner:v5>
    šŸŒŸ  Enabled addons: storage-provisioner, default-storageclass
    šŸ„  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
    I think this much memory can suffice
    Yet the pod with agent is not able to start and fails again & again
    Anna Geller

    Anna Geller

    4 months ago
    I see you are running it on Ubuntu 18.04 - do you have a chance to use Ubuntu 20.04 instead?
    Kayvan Shah

    Kayvan Shah

    4 months ago
    Have got ubuntu 20.04 lts can try running with wsl2
    Anna Geller

    Anna Geller

    4 months ago
    What's your current setup - is your end goal running Prefect on Windows? if so, we are working on this
    Kayvan Shah

    Kayvan Shah

    4 months ago
    Deploying prefect flows and workloads on AWS EKS. Currently working on POC on a company's on-premise Ubuntu 18.04 LTS server
    Anna Geller

    Anna Geller

    4 months ago
    in that case doing PoC on a small single node K8s cluster on AWS EKS may be easier, but definitely your choice how you decide to approach it, there's no right or wrong here - the link I shared is for AWS EKS
    Kayvan Shah

    Kayvan Shah

    4 months ago
    Even if I am running Docker and minikube on ubuntu 18.04 it shouldn't make much difference as prefect2b5 image may be built upon latest linux stable version?
    Though the minikube deployment did not run as expected I was able to run most of the things locally
    Anna Geller

    Anna Geller

    4 months ago
    the consultant answer: it depends šŸ™‚
    e.g. you may need to install Prefect to create deployments from this server - so you would need to install your orchestration layer e.g. on minikube and then the environment from which you create deployments would also need a 2.0 installation - if you install it on the same VM, then it's easier if it'S Ubuntu 20.04 than 18.04