Mia
04/05/2022, 2:12 PMFailed to load and execute flow run: ModuleNotFoundError("No module named 'pandas'")
I don’t think it’s coming from missing pandas module. When I log into my docker image, I have no problem importing pandas. I’ve installed all the required packages in the base environment. I’ve also tried loading all the packages in the conda environment and that the conda environment is loaded upon docker image is loaded.
Is this some python path issue? What I don’t understand is that i’ve done this several times before and i didn’t have this issue until nowAnna Geller
Kevin Kho
conda activate
doesn’t workMia
04/05/2022, 2:28 PMFROM prefecthq/prefect:latest
WORKDIR $HOME/myproject
ADD . .
SHELL [ "/bin/bash", "--login", "-c" ]
RUN apt-get clean && apt-get update && apt-get install -y curl vim
ENV MINICONDA_VERSION latest
ENV CONDA_DIR $HOME/miniconda3
RUN curl -LO "<https://repo.anaconda.com/miniconda/Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh>" \
&& bash Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh -p $CONDA_DIR -b
RUN rm Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh
ENV PATH=$CONDA_DIR/bin:$PATH
RUN echo ". $CONDA_DIR/etc/profile.d/conda.sh" >> ~/.profile
RUN conda init bash
RUN conda env update -n base -f environment.yml --prune && \
conda clean --all -f -y
This is the copy of my DockerfileKevin Kho
Mia
04/05/2022, 3:38 PMKevin Kho
bash
?Mia
04/05/2022, 4:23 PMKevin Kho
Mia
04/05/2022, 4:26 PMKevin Kho
Mia
04/05/2022, 7:42 PMKevin Kho
#!/bin/bash --login
# The --login ensures the bash configuration is loaded,
# enabling Conda.
# Enable strict mode.
set -euo pipefail
# ... Run whatever commands ...
# Temporarily disable strict mode and activate conda:
set +euo pipefail
conda activate myenv
# Re-enable strict mode:
set -euo pipefail
Mia
04/05/2022, 8:24 PM(dockerimage)# prefect execute flow-run
No module named 'pandas'
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/cli/execute.py", line 92, in flow_run
raise exc
File "/usr/local/lib/python3.7/site-packages/prefect/cli/execute.py", line 73, in flow_run
flow = storage.get_flow(flow_data.name)
File "/usr/local/lib/python3.7/site-packages/prefect/storage/gitlab.py", line 106, in get_flow
file_contents=contents.decode(), flow_name=flow_name
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/storage.py", line 89, in extract_flow_from_file
exec(contents, exec_vals)
File "<string>", line 7, in <module>
ModuleNotFoundError: No module named 'pandas'
the python version where i have everything installed should be python 3.9 and not 3.7Kevin Kho
Mia
04/05/2022, 8:27 PMprefect execute flow-run
uses a specific python version and not the one in the pathKevin Kho
Mia
04/05/2022, 8:50 PMwhich python
, i get miniconda python path. In miniconda python path, i have all the packages i need installed as expectedAnna Geller