Ethan Veres
09/05/2023, 3:43 PMMarvin
09/05/2023, 3:44 PMprefect.yml
file.
For the build step, you could use the run_shell_script
step and feed the output into the build_docker_image
step. Here's a little example to brighten your day:
yaml
build:
- prefect.deployments.steps.run_shell_script:
id: get-commit-hash
script: git rev-parse --short HEAD
stream_output: false
- prefect_docker.deployments.steps.build_docker_image:
requires: prefect-docker
image_name: my-image
image_tag: "{{ get-commit-hash.stdout }}"
dockerfile: auto
The id
field is used in the run_shell_script
step so that its output can be referenced in the next step.
The push section is most critical for situations in which code is not stored on persistent filesystems or in version control. In this scenario, code is often pushed and pulled from a Cloud storage bucket of some kind (e.g., S3, GCS, Azure Blobs, etc.). The push section allows users to specify and customize the logic for pushing this code repository to arbitrary remote locations.
The pull section is the most important section within the prefect.yaml
file as it contains instructions for preparing your flows for a deployment run. These instructions will be executed each time a deployment created within this folder is run via a worker.
There are three main types of steps that typically show up in a pull section:
- `set_working_directory`: this step simply sets the working directory for the process prior to importing your flow
- `git_clone`: this step clones the provided repository on the provided branch
- `pull_from_{cloud}`: this step pulls the working directory from a Cloud storage location
I hope this information is helpful, or at least not too depressing. For more details, you might want to visit the Prefect documentation. It's a riveting read, I assure you.Ethan Veres
09/05/2023, 3:50 PM