Is there a way to build deployments without having...
# prefect-community
h
Is there a way to build deployments without having to install all dependencies? I’m trying to set up our CD pipeline for deploying our flows on ECS infrastructure and it looks like I have to install all the dependencies and set environment variables correctly to have
deployment build
work with an ecs-task infra block. This looks related but only looks like it pertains to the docker-container infra block. Otherwise we can manually create the deployments for now. We are following a similar pipeline to this one: https://github.com/anna-geller/dataflow-ops/blob/main/.github/workflows/ecs_prefect_agent.yml
a
in theory, you could but you would need to move e.g. the imports to the tasks
the issue here is more one related to Python -- we need to evaluate the flow function to build a parameter schema for a deployment, but to do that, we need to read the module (entire script) -- moving the imports would work, but easier is to e.g. run CI/CD within the image that you already build for your infra or to combine deployment build within the same CI/CD job/step that builds the image
👍 1
t
We do what Anna suggested above: Build and push the Docker container in CI (containing prefect, dependencies, and flow code). And then immediately afterwards run
docker run --rm -i [imagewejustbuilt] python3 deploy.py
. Where
deploy.py
is some script that imports the flow and calls
Deployment.build_from_flow
.
👍 1
🙌 2
h
Okay that makes sense…I’m also wondering if it’s possible in our case to omit the command (ex.
prefect deployment build -n flow_123 -q dev -ib ecs-task/dev -a flows/my_flow.py:my_flow --skip-upload
) from the CI/CD altogether. My thinking was: 1. Developer adds new flow, runs commands to create deployments on their local machines 2. From that point on, the docker image is built as part of cd and the infra blocks are updated to point to new image versions Is this a valid approach in our case? It seems similar to the CI/CD for just a docker image https://github.com/anna-geller/prefect-docker-deployment/blob/main/.github/workflows/docker_image.yaml since we aren’t uploading anything to S3. Thank you both for your responses!
c
You can omit the deployment build in your ci/cd pipeline if a developer runs it locally. Running the deployment build / apply updates it server side, then the agent will retrieve the flow run that it creates
h
Great, thanks!