Hi everyone, we are using Prefect Cloud and Azure ...
# ask-community
k
Hi everyone, we are using Prefect Cloud and Azure ACI's for the workers. For the ACI command for install I wanted to confirm I just need
/bin/bash -c 'pip install prefect-azure && prefect worker start --pool <work-pool-name> --type azure-container-instance'
and for flows dependencies are setup on a per flow basis? i.e. I think in prefect 2 I installed all possible extra packages on the agent and flows used those. I have been having success configuring the job variables in the cloud UI and PIP EXTRA PACKAGES but I wanted to confirm that's the most efficient way. Any help much appreciated!
b
Hey Keith! Yes, to start the worker process that command is all you need. If you don't have many dependencies, and don't expect them to change often, you could keep setting them through the Cloud UI and
job_variables.
If your dependencies change down the line, however, you would need to remember to go and manually change the packages you defined. For managing flow dependencies, typically we recommend creating a docker image that has all of them baked in (especially if you're using a containerized work pool type).
If you're using
.deploy()
it will build a docker image that contains your flow code and dependencies (the ones listed out in your
requirements.txt
file) by default.
k
our team is using a git repo for flow code storage not on custom docker images yet, can we specify the pip extra packages in our deployment code?
b
Yup, something like this: if name == "__main__": get_repo_info.deploy( name="my-deployment-never-pull", work_pool_name="my-docker-pool", job_variables={"env": {"EXTRA_PIP_PACKAGES": "boto3"} }, ... )
Or if you're deploying through the cli with
prefect deploy
, you can set a
pull
step in your
prefect.yaml
that installs dependencies after cloning the repo
Copy code
pull:
- prefect.deployments.steps.git_clone:
    id: clone-step # needed to be referenced in subsequent steps
    repository: <https://github.com/org/repo.git>
- prefect.deployments.steps.pip_install_requirements:
    directory: "{{ clone-step.directory }}" # `clone-step` is a user-provided `id` field
    requirements_file: requirements.txt
k
Thanks Bianca, it's starting to make sense. I think I was confusing prefect 3 and 2 agents/workers
with ACI being serverless