Nicholas Pini
11/29/2024, 12:05 PM/utils
, which is structured like this:
utils
|_ src
|_ utils
|_ __init__.py
source code...
My flow gets pulled into the /flow
folder from S3. The problem is that I can only use utils
if I explicitly add utils/src/
to sys.path
, otherwise I tried:
• installing utils
with pip install -e utils
. I did this on my local machine, and this automatically adds utils/src
to sys.path
. This does not happen when using prefect.deployments.steps.run_shell_script
• setting PYTHONPATH
to utils/src
. Doing this makes it so that sys.path
contains /utils/src
, but importing the module fails anyway. This is what I don't get: I end up with the same exact sys.path as I have when I manually tweak it, but it fails anyway. Note that I tried both setting PYTHONPATN
in my Dockerfile, and passing it as a job variable.
Note that the error happens here:
File "/opt/venv/lib/python3.10/site-packages/prefect/utilities/importtools.py", line 205, in import_object
module = load_script_as_module(script_path)