Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

How do i setup the command in docker-container block to first it would run the requirements.txt file that the flow run gets from azure blob, before running the prefect engine. Following doesn't work

I would install the dependencies in your Dockerfile
```FROM yourbaseimage

COPY requirements.txt .

RUN pip install -r requirements.txt```


<@U02GMEZU18B> I do that, but we want a base dockerfile that can be built upon on individual flows to keep the base clean and only add the specific requirements in each container. That way we are more secure in not running into package depency issues if different projects runs different versions of same package

The flow is then grabbed from the blob storage, keeping flow and container independent of each other. So we're looking for a way to do this

You could debug this locally by attempting to do a `DockerContainer(command=[…]).run()`

To run multiple commands, you’ll need to do a bit more

e.g.
```DockerContainer(command=["bash", "-c", "pip install cowsay &amp;&amp; echo hi!"])```

solved the several commands problem, issue is that the requirements from blob storage does not get pulled into the docker container prior to running python -m -prefect.engine, so i can't run the command prior.

Is the best practice really to have a seperate image per flow, or one standard image that all your flows use?

It depends if you’d prefer a slowdown at build time or runtime

In theory, with an image per flow, images will all share common base layers and you can use Docker layer caching to prevent duplicate data transfer.

We’re likely to allow Python dependencies to be installed by the Prefect engine before executing your code sometime in the future.

I will bring this to the attention of the team after the weekend and see what we decide upon for now - thanks for the insight!

Since you're using `docker-container`, It sounds like the `EXTRA_PIP_PACKAGES` environment variable might help, assuming you are using one of the Prefect base images directly or building atop one?

<https://docs.prefect.io/concepts/infrastructure/#installing-extra-dependencies-at-runtime|As the docs note>, installing the packages at runtime adds overhead and has disadvantages, but you are the best judge of whether it makes sense for your workflow.

Setting that environment variable (either during deployment or afterward via the block UI) is a bit less convenient than running pip against requirements.txt, but it might be workable? If you want to try it, you can probably flatten requirements.txt into a `pip install` compatible string you can put in the environment variable by running `awk '{gsub(" ","",$0);printf("%s ",$0)}' requirements.txt` if you're running Linux or MacOS.