What if I have an existing code base (for machine learning) with 4 separate steps; 1. Data preprocess 2. Train 3. Inference 4. Visualizations And I want to introduce Prefect to it but still allow others without prefect installed to run the code?

Not answering your question but: IMO, if you are using other python packages than prefect, very likely they will need to

pip install

something else as well already. Maybe a better approach could be to use poetry or even a Docker image, so everybody has the same environment. But your use case might not apply to this 🙂

Thanks @Joao Moniz, what’s the difference between this and virtualenv?

With the Docker image you can guarantee that everybody will use the exact same python environment (python version and have the same dependencies). Instead of using a virtual environment you will run the docker image. Poetry is more like a package manager (like

pip

) but with some advantages like avoiding dependencies conflict between different python packages. It's very easy to install and to use, and poetry will create a virtual environment by default for each project that you have (you can use this option to make it in-project). The only thing that poetry won't do is to manage the Python version your team is using, for that you can use

pyenv

for example if needed. So you can use only Poetry, you can use only Docker, or you can use Docker + Poetry. Whatever fits better for your project/team needs

Gotcha, thanks for the detailed explanation!