Hello everyone! Is the `EXTRA_PIP_PACKAGES` enviro...
# prefect-community
a
Hello everyone! Is the
EXTRA_PIP_PACKAGES
environment variable supposed to work with
KubernetesFlowRunner
?
k
Looks like it should work if you use the base Prefect image to build your image because itโ€™s in the entrypoint. KubernetesFlowRunner can take in an
env
Not 100% sure. What did you try?
I think this syntax will work
Copy code
KubernetesFlowRunner(env={"EXTRA_PIP_PACKAGES": "my-extra-package1 my-extra-package2"})
a
I use the following command but it doesn't work
KubernetesFlowRunner(image="prefecthq/prefect:2.0b2-python3.8", env={"EXTRA_PIP_PACKAGES": "pandas"})
. Could this behavior be caused by redefining the
entrypoint
in Kubernetes Job manifest somewhere in Prefect?
k
Are you saying you added an entrypoint? Or do you mean Prefect might be overriding it to deploy a Flow?
a
I'm not sure
EXTRA_PIP_PACKAGES
works with KubernetesFlowRunner. I'm sure it works with DockerFlowRunner though.
a
@Kevin Kho yes, second option
k
Ok let me try one sec
a
Some output from
kubectl describe pods ...
:
Copy code
Containers:
  job:
    Container ID:  <docker://563d2687bfc19688248cb5e6d7f6bd053f7a09d181b066be50d5dd021aac79d>5
    Image:         prefecthq/prefect:2.0b2-python3.8
    Image ID:      <docker-pullable://prefecthq/prefect@sha256:6c7e1cab30ed20a10c557e8c2f832582326b4d00fda53d3cd1e70073dc535b65>
    Port:          <none>
    Host Port:     <none>
    Command:
      python
      -m
      prefect.engine
      f52ed613-33a4-4b28-a8c8-2f8f2b222a00
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 21 Mar 2022 20:12:05 +0000
      Finished:     Mon, 21 Mar 2022 20:12:07 +0000
    Ready:          False
    Restart Count:  0
    Environment:
      EXTRA_PIP_PACKAGES:  pandas
      PREFECT_API_URL:     <http://orion:4200/api>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cwmkj (ro)
@Anna Geller yes, it works for me in
DockerFlowRunner
case :)
k
Actually this will take a while for me to try. Will get back to you tonight
a
ok, @Anna Geller @Kevin Kho thanks for the super fast response :)
๐Ÿ‘ 1
a
@Anatoly Myachev just tested it with
KubernetesFlowRunner
-
EXTRA_PIP_PACKAGES
doesn't work. So far you can only use
EXTRA_PIP_PACKAGES
with the
DockerFlowRunner
it seems. Let me open an issue to check if we can add it to
KubernetesFlowRunner
as well. @Marvin open "Orion: allow to install EXTRA_PIP_PACKAGES at runtime using KubernetesFlowRunner"
๐Ÿ‘€ 1
431 Views