<@ULVA73B9P> how would I deploy some images on doc...
# ask-marvin
f
@Marvin how would I deploy some images on dockerhub as workflow? This is my current github action
Copy code
- name: Create deployment summary
        run: |
          echo "## 🚀 Prefect Individual Docker Deployment Summary" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "**Version:** \`${{ env.VERSION }}\`" >> $GITHUB_STEP_SUMMARY
          echo "**Short Version:** \`${{ env.SHORT_VERSION }}\`" >> $GITHUB_STEP_SUMMARY
          echo "**Commit:** \`${{ github.sha }}\`" >> $GITHUB_STEP_SUMMARY
          echo "**Branch:** \`${{ github.ref_name }}\`" >> $GITHUB_STEP_SUMMARY
          echo "**Prefect UI:** [<http://IP:4200>](<http://IP:4200>)" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "### Deployed Flows (Individual Docker Images):" >> $GITHUB_STEP_SUMMARY
          echo "- test-deployment: \`${{ env.DOCKER_REGISTRY }}/${{ env.IMAGE_PREFIX }}-test:${{ env.VERSION }}\`" >> $GITHUB_STEP_SUMMARY
m
✅ thought for 413.1 seconds
Great question. Here’s a simple, repeatable way to run Prefect flows from Docker images you push to Docker Hub, using a Docker work pool and a GitHub Actions workflow. High-level flow - Build and push your Docker images to Docker Hub (one image per flow, if you prefer). - Register deployments that reference those images. - Run a Docker worker that pulls from your work pool and executes the containers. 1) Create a Docker work pool and start a worker Run these once (or manage via IaC):
Copy code
# Create a Docker work pool
prefect work-pool create my-docker-pool --type docker

# Start a worker on a machine that has access to Docker
# Tip: install docker extras: pip install "prefect[docker]"
prefect worker start --pool my-docker-pool
2) Define deployments that point to your images Preferred: define deployments in prefect.yaml (checked into your repo). Example with two flows each using a different image:
Copy code
name: my-project
prefect-version: 3.0.0

deployments:
- name: test-deployment
  entrypoint: flows/test.py:test_flow
  work_pool:
    name: my-docker-pool
    job_variables:
      image: your-dockerhub-org/your-image-test:1.2.3
      env:
        PREFECT_LOGGING_LEVEL: INFO

- name: other-deployment
  entrypoint: flows/other.py:other_flow
  work_pool:
    name: my-docker-pool
    job_variables:
      image: your-dockerhub-org/your-image-other:1.2.3
Then register them from your repo root:
Copy code
prefect deploy
Alternative: pass image per deployment at deploy time If you want to use one prefect.yaml and set the image dynamically (e.g., by tag in CI), you can override the image per deployment during deploy with job variables. For each deployment, run:
Copy code
prefect deploy ./flows/test.py:test_flow \
  --name test-deployment \
  --pool my-docker-pool \
  --job-variable image=your-dockerhub-org/your-image-test:${VERSION}
3) GitHub Actions example Matrix-build multiple images, push to Docker Hub, then register deployments pointing at those tags: ``` name: Build and Deploy Flows on: push: branches: [ main ] env: DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }} DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }} VERSION: ${{ github.sha }} SHORT_VERSION: ${{ github.sha:07 }} IMAGE_PREFIX: your-dockerhub-org/prefect PREFECT_API_URL: ${{ secrets.PREFECT_API_URL }} # Cloud or self-hosted PREFECT_API_KEY: ${{ secrets.PREFECT_API_KEY }} # Cloud key, if applicable jobs: build-push-deploy: runs-on: ubuntu-latest strategy: matrix: flow: - name: test-deployment image_suffix: test entrypoint: flows/test.py:test_flow - name: other-deployment image_suffix: other entrypoint: flows/other.py:other_flow steps: - uses: actions/checkout@v4 - name: Log in to Docker Hub run: echo "$DOCKERHUB_TOKEN" | docker login -u "$DOCKERHUB_USER" --password-stdin - name: Build image run: | IMAGE=${IMAGE_PREFIX}-${{ matrix.flow.image_suffix }}:${VERSION} docker build \ -f Dockerfile \ --build-arg PIP_EXTRA_INDEX_URL=${{ secrets.PIP_EXTRA_INDEX_URL }} \ -t $IMAGE . echo "IMAGE=$IMAGE" >> $GITHUB_ENV - name: Push image run: | docker push $IMAGE - name: Set up Python and install Prefect uses: actions/setup-python@v5 with: python-version: "3.11" - run: pip install "prefect[docker]" # If using Prefect Cloud - name: Prefect login (Cloud) if: env.PREFECT_API_KEY != '' run: prefect cloud login --key "$PREFECT_API_KEY" --account default --workspace default # If using self-hosted Prefect Server, ensure PREFECT_API_URL is set in env - name: Deploy Prefect deployment referencing the new image working-directory: . run: | prefect deploy "${{ matrix.flow.entrypoint }}" \ --name "${{ matrix.flow.name }}" \
--pool "my-docker-pool" \ --job-variable image=$IMAGE - name: Create deployment summary run: | echo "## 🚀 Prefect Individual Docker Deployment Summary" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY echo "*Version:* \`${{ env.VERSION }}\`" >> $GITHUB_STEP_SUMMARY echo "*Short Version:* \`${{ env.SHORT_VERSION }}\`" >> $GITHUB_STEP_SUMMARY echo "*Commit:* \`${{ github.sha }}\`" >> $GITHUB_STEP_SUMMARY echo "*Branch:* \`${{ github.ref_name }}\`" >> $GITHUB_STEP_SUMMARY echo "*Prefect UI:* http://IP:4200" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY echo "### Deployed Flows (Individual Docker Images):" >> $GITHUB_STEP_SUMMARY echo "- ${{ matrix.flow.name }}: \`${IMAGE}\`" >> $GITHUB_STEP_SUMMARY ``` Notes and gotchas - Run the Docker worker on infrastructure that can reach your Docker daemon and pull from Docker Hub. - Make sure your images contain your code + dependencies and an appropriate entrypoint for your flow when Prefect invokes it. - You can set a default image at the work pool level by editing the pool’s base job template, but per-deployment overrides with job variables are usually clearer in CI. - If you prefer a purely declarative approach, pin the image in prefect.yaml and just run
prefect deploy
after pushing images. Helpful docs - Prefect 3: prefect.yaml deployments - Scalable microservices orchestration with Prefect and Docker - Work pools and workers overview If you share: - your flow entrypoints, and - how you want to tag images (versioning), I can tailor the prefect.yaml and the GitHub Actions matrix exactly to your setup.