Nicholas Pini
11/29/2024, 12:05 PM/utils, which is structured like this:
utils
|_ src
|_ utils
|_ __init__.py
source code...
My flow gets pulled into the /flow folder from S3. The problem is that I can only use utils if I explicitly add utils/src/ to sys.path , otherwise I tried:
• installing utils with pip install -e utils . I did this on my local machine, and this automatically adds utils/src to sys.path. This does not happen when using prefect.deployments.steps.run_shell_script
• setting PYTHONPATH to utils/src . Doing this makes it so that sys.path contains /utils/src , but importing the module fails anyway. This is what I don't get: I end up with the same exact sys.path as I have when I manually tweak it, but it fails anyway. Note that I tried both setting PYTHONPATN in my Dockerfile, and passing it as a job variable.
Note that the error happens here:
File "/opt/venv/lib/python3.10/site-packages/prefect/utilities/importtools.py", line 205, in import_object
module = load_script_as_module(script_path)