Zheng Xie

    Zheng Xie

    9 months ago
    I can get Prefect agent running locally and triggered by Prefect Cloud UI. Now I want to move the agent to a Kubernetes cluster on IBM Cloud. There is not a tutorial to list the steps and the docs from Prefect is grouped around some concepts that I don’t know what I need to accomplish the goal. Can you please list the steps? for example:1. set flow storage to docker 2. Push the code to container registry in the kubernetes cluster 3. … I am hoping with this list, I can look up the Prefect docs to figure out what to do.
    Anna Geller

    Anna Geller

    9 months ago
    @Zheng Xie Here are some docs that show how to set this up:1. https://docs.prefect.io/orchestration/execution/storage_options.html#docker 2. https://docs.prefect.io/orchestration/agents/kubernetes.html#requirements In general, you need a YAML deployment file to deploy your agent to the IBM Kubernetes cluster. You can generate it:
    prefect agent kubernetes install -k YOUR_API_KEY --rbac >> prefect_agent.yaml
    then you can customize this file and apply it to the cluster:
    kubectl apply -f prefect_agent.yaml
    Zheng Xie

    Zheng Xie

    9 months ago
    Thanks Anna. On flow.storage.build() (this is docker storage) I get the following error: File “/Users/zhengxie1/workspace/cog_ana_model_train/venv/lib/python3.7/site-packages/prefect/storage/docker.py”, line 585, in push_image raise InterruptedError(line.get(“error”)) InterruptedError: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active. How do I provide credential please?
    Anna Geller

    Anna Geller

    9 months ago
    it’s usually something like:
    docker login -u $REGISTRY_USER -p $REGISTRY_PASSWORD $REGISTRY_URL
    OK, IBM seems to be a bit more “special” 😄
    ibmcloud cr login
    check out this doc: https://cloud.ibm.com/docs/Registry?topic=Registry-getting-started
    Zheng Xie

    Zheng Xie

    9 months ago
    I can use CLI to push image. The question is can I run python code using prefect library to build the image locally, then I use IBM cloud CLI to push it into the registry?
    flow.storage.build() seems to run behind the scene, it builds and push the image and I don’t know how to provide credential in python runtime.
    Anna Geller

    Anna Geller

    9 months ago
    if you login to IBM Container Registry from a terminal session, and from the same session you run:
    prefect register --project yourproject -p yourflow.py
    then it will work just fine, you should be able to push the image
    Zheng Xie

    Zheng Xie

    9 months ago
    Hi Anna, I can push to icr now. Thanks a lot. I did see Docker storage before your tips, but it is listed under “Legacy environment”, so I wasn’t sure if was applicable.
    Anna Geller

    Anna Geller

    9 months ago
    @Zheng Xie you’re absolutely right! thanks for pointing that out! Something must have gone wrong with the docs structure, none of those storage classes belong to legacy environments. Thanks again!