Been toying with some organizational strategies for our pipelines. I like using a

src/

folder with the various components in there like

src/flows/

and

src/mylib/

. When deploying I run into import issues (e.g.

flows/flow_a.py

trying to import

mylib

). I can solve this locally by altering PYTHONPATH, but I want it to also work in the execution environment. Would adding an

infra_override

to each deployment like below work? Are there better ways of going about this?

infra_overrides: {
    "env.PYTHONPATH": "src/"
  }

i like to just use

pyproject.toml

and package up my project so i can

pip install .

or

pip install ".[whatever]"

like this, and then just import like a package (bc it is one) or is there some nuance I'm missing where you really need to mess with

PYTHONPATH

?

Only nuance is I was using a git-docker recipe to clone the project into an existing Docker image. I guess I can pivot to a Docker-only pattern that rebuilds the image with each change.

> Only nuance is I was using a git-docker recipe to clone the project into an existing Docker image. thats what I like to do as well - i (personally) often dont like rebuilding src into my images each time bc my src changes a lot more than my deps (although i know there are cases where baking it all in is necessary) you could

pip install .

your lib in your pull step with

run_shell_script

or use

EXTRA_PIP_PACKAGES

Touche. Yeah I'll try that, see how it goes. Thanks!

👍