Thread
#prefect-community
    a

    Anatoly Myachev

    6 months ago
    Hello everyone! Is the
    EXTRA_PIP_PACKAGES
    environment variable supposed to work with
    KubernetesFlowRunner
    ?
    Kevin Kho

    Kevin Kho

    6 months ago
    Looks like it should work if you use the base Prefect image to build your image because it’s in the entrypoint. KubernetesFlowRunner can take in an
    env
    Not 100% sure. What did you try?
    I think this syntax will work
    KubernetesFlowRunner(env={"EXTRA_PIP_PACKAGES": "my-extra-package1 my-extra-package2"})
    a

    Anatoly Myachev

    6 months ago
    I use the following command but it doesn't work
    KubernetesFlowRunner(image="prefecthq/prefect:2.0b2-python3.8", env={"EXTRA_PIP_PACKAGES": "pandas"})
    . Could this behavior be caused by redefining the
    entrypoint
    in Kubernetes Job manifest somewhere in Prefect?
    Kevin Kho

    Kevin Kho

    6 months ago
    Are you saying you added an entrypoint? Or do you mean Prefect might be overriding it to deploy a Flow?
    Anna Geller

    Anna Geller

    6 months ago
    I'm not sure
    EXTRA_PIP_PACKAGES
    works with KubernetesFlowRunner. I'm sure it works with DockerFlowRunner though.
    a

    Anatoly Myachev

    6 months ago
    @Kevin Kho yes, second option
    Kevin Kho

    Kevin Kho

    6 months ago
    Ok let me try one sec
    a

    Anatoly Myachev

    6 months ago
    Some output from
    kubectl describe pods ...
    :
    Containers:
      job:
        Container ID:  <docker://563d2687bfc19688248cb5e6d7f6bd053f7a09d181b066be50d5dd021aac79d>5
        Image:         prefecthq/prefect:2.0b2-python3.8
        Image ID:      <docker-pullable://prefecthq/prefect@sha256:6c7e1cab30ed20a10c557e8c2f832582326b4d00fda53d3cd1e70073dc535b65>
        Port:          <none>
        Host Port:     <none>
        Command:
          python
          -m
          prefect.engine
          f52ed613-33a4-4b28-a8c8-2f8f2b222a00
        State:          Terminated
          Reason:       Completed
          Exit Code:    0
          Started:      Mon, 21 Mar 2022 20:12:05 +0000
          Finished:     Mon, 21 Mar 2022 20:12:07 +0000
        Ready:          False
        Restart Count:  0
        Environment:
          EXTRA_PIP_PACKAGES:  pandas
          PREFECT_API_URL:     <http://orion:4200/api>
        Mounts:
          /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cwmkj (ro)
    @Anna Geller yes, it works for me in
    DockerFlowRunner
    case πŸ˜ƒ
    Kevin Kho

    Kevin Kho

    6 months ago
    Actually this will take a while for me to try. Will get back to you tonight
    a

    Anatoly Myachev

    6 months ago
    ok, @Anna Geller @Kevin Kho thanks for the super fast response πŸ˜ƒ
    Anna Geller

    Anna Geller

    6 months ago
    @Anatoly Myachev just tested it with
    KubernetesFlowRunner
    -
    EXTRA_PIP_PACKAGES
    doesn't work. So far you can only use
    EXTRA_PIP_PACKAGES
    with the
    DockerFlowRunner
    it seems. Let me open an issue to check if we can add it to
    KubernetesFlowRunner
    as well. @Marvin open "Orion: allow to install EXTRA_PIP_PACKAGES at runtime using KubernetesFlowRunner"
    Marvin

    Marvin

    6 months ago