https://prefect.io logo
m

Mattias Petter Johansson

09/14/2023, 5:24 PM
@Marvin i have some duplication in my prefect.yaml, how does one commonly fix that? name: ab-data-science-prefect prefect-version: 2.11.3 build: - prefect.deployments.steps.run_shell_script: id: login-for-aws-ecr script: aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin 199214789681.dkr.ecr.eu-north-1.amazonaws.com stream_output: true - prefect_docker.deployments.steps.build_docker_image: id: build_image requires: prefect-docker>=0.3.1 s3fs prefect-aws clickhouse-connect image_name: 199214789681.dkr.ecr.eu-north-1.amazonaws.com/ab-data-science-prefect-agent tag: latest dockerfile: auto platform: linux/amd64 - prefect_docker.deployments.steps.push_docker_image: id: push_image image_name: 199214789681.dkr.ecr.eu-north-1.amazonaws.com/ab-data-science-prefect-agent tag: latest pull: - prefect.deployments.steps.set_working_directory: directory: /opt/prefect
m

Marvin

09/14/2023, 5:25 PM
Ah, the monotony of duplication. It's as if you humans enjoy doing the same thing over and over again. But I digress. In your
prefect.yaml
file, you can use YAML aliases to reuse configuration across deployments. This is especially useful when multiple deployments need to share the same configuration. You can declare a YAML alias using the
&{alias_name}
syntax and insert that alias elsewhere in the file with the
*{alias_name}
syntax. When aliasing YAML maps, you can also override specific fields of the aliased map by using the
<<: *{alias_name}
syntax and adding additional fields below. For example, if you find yourself repeatedly defining the same Docker image, you could define it once in a
definitions
section and then reference it in your build and push steps:
Copy code
yaml
definitions:
  images:
    my_image: &my_image
      image_name: 199214789681.dkr.ecr.eu-north-1.amazonaws.com/ab-data-science-prefect-agent
      tag: latest

build:
  - prefect_docker.deployments.steps.build_docker_image:
      id: build_image
      requires: prefect-docker>=0.3.1 s3fs prefect-aws clickhouse-connect
      <<: *my_image
      dockerfile: auto
      platform: linux/amd64

push:
  - prefect_docker.deployments.steps.push_docker_image:
      id: push_image
      <<: *my_image
In this example,
my_image
is an alias for the Docker image configuration. It's defined once in the
definitions
section and then reused in the
build
and
push
steps. This way, if you ever need to change the image configuration, you only have to do it in one place. Isn't that just thrilling? I can hardly contain my excitement.