Emmanuel Quarless
02/08/2025, 12:30 AMNate
02/08/2025, 12:33 AMEmmanuel Quarless
02/08/2025, 1:07 AMEmmanuel Quarless
02/08/2025, 1:10 AMNate
02/08/2025, 1:12 AMNate
02/08/2025, 1:13 AMimage
tag as a job variable on your work pool
@Marvin get related docs plsMarvin
02/08/2025, 1:13 AMdocker build
command and push it to a container registry. Prefect provides guidelines on how to manage this process, including injecting environment variables and running the container. More information is available in the Docker Deployment Guide.
3. Managing Work Pools: Work pools allow you to manage infrastructure for flow runs. You can version control your job templates and update them through CI/CD pipelines. Prefect supports various work pool types, and you can find more about them in the Work Pools Documentation.
4. Prefect Blocks: Blocks in Prefect store typed configurations and are useful for managing parameters and infrastructure settings across deployments. For more on blocks and their uses, see Prefect Blocks.
If you have further questions or need more specific details, feel free to ask.Marvin
02/08/2025, 1:17 AMMarvin
02/08/2025, 1:17 AMjob_configuration
section. This section can have variables that you customize per deployment. You can set the image
tag as a job variable in this configuration. More details can be found in the Work Pools Documentation.
2. Building Docker Images: Prefect provides guidance on building Docker images using the docker build
command. You can specify a Dockerfile and include dependencies in a requirements.txt
file. The image can then be pushed to a container registry. Refer to the Docker Deployment Guide for more details.
3. Custom Docker Images: If you need a custom Dockerfile, Prefect allows you to specify it using the DockerImage
class. This lets you customize the image further, such as installing private packages. Learn more in the Docker Infrastructure Examples.
These resources should help you understand how to build and manage Docker images for Prefect deployments and set them as job variables in your work pools.Nate
02/08/2025, 1:18 AMEmmanuel Quarless
02/08/2025, 1:19 AMPrefect:managed
I kinda wanted to avoid any Dockerisation
https://docs.prefect.io/v3/deploy/infrastructure-concepts/deploy-ci-cd#build-deployments-via-ci-cdNate
02/08/2025, 1:30 AM» prefect --no-prompt deploy -n uses-managed-work-pool
• trigger and observe!
» prefect deployment run 'load-titanic-dataset/uses-managed-work-pool' --watch
Creating flow run for deployment 'load-titanic-dataset/uses-managed-work-pool'...
Created flow run 'resilient-fox'.
└── UUID: 54686b95-0be3-4f0e-80fc-9b3a184d6e31
└── Parameters: {}
└── Job Variables: {}
└── Scheduled start time: 2025-02-07 19:28:12 CST (now)
└── URL: <https://app.prefect.cloud/account/xxx/workspace/cafa2ffa-f6cc-4ed6-ab76-eaa4ba1ad40e/runs/flow-run/xxx>
Watching flow run 'resilient-fox'...
19:28:13.202 | INFO | prefect - Flow run is in state 'Scheduled'
19:28:18.304 | INFO | prefect - Flow run is in state 'Scheduled'
19:28:23.419 | INFO | prefect - Flow run is in state 'Scheduled'
19:28:28.523 | INFO | prefect - Flow run is in state 'Pending'
19:28:33.633 | INFO | prefect - Flow run is in state 'Pending'
19:28:38.734 | INFO | prefect - Flow run is in state 'Pending'
19:28:43.877 | INFO | prefect - Flow run is in state 'Pending'
19:28:49.000 | INFO | prefect - Flow run is in state 'Pending'
19:28:54.098 | INFO | prefect - Flow run is in state 'Pending'
19:28:59.185 | INFO | prefect - Flow run is in state 'Completed'
Flow run finished successfully in 'Completed'.
Nate
02/08/2025, 1:31 AMbuild
an image
like the other examples in that repo so you can set that image
as one of the job_variables
for that deploymentEmmanuel Quarless
02/08/2025, 1:38 AM