I have a simple Github CI file with ```jobs: bu...
# ask-community
c
I have a simple Github CI file with
Copy code
jobs:
  build:
    name: Deploy Prefect with Modal Work Pool
    runs-on: ubuntu-latest
    environment: Development
    env:
      MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}
      MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}

    steps:
      - uses: actions/checkout@v4

      - name: Install Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Prefect Auth
        uses: PrefectHQ/actions-prefect-auth@v1
        with:
          prefect-api-key: ${{ secrets.PREFECT_API_KEY }}
          prefect-workspace: ${{ secrets.PREFECT_WORKSPACE }}

      - name: Create Work Pool
        run: echo "y" | prefect work-pool create --type modal:push --provision-infra modal-pool || true

      - name: Prefect Deploy
        run: prefect deploy
and a prefect.yaml of
Copy code
pull:
  - prefect.deployments.steps.git_clone:
      repository: redacted
      branch: feature/prefect-modal-testing
      access_token: "{{ secrets.GITHUB_TOKEN }}"

build:
  - prefect_docker.deployments.steps.build_docker_image:
      requires: prefect-docker
      image_name: arnold/base
      tag: dev
      dockerfile: dockerfiles/Dockerfile.service

push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker
      image_name: "{{ build-image.image_name }}"
      tag: "{{ build-image.tag }}"
      credentials: "{{ prefect.blocks.docker-registry-credentials.dev-registry }}"
      additional_tags: "{{ build-image.additional_tags }}"

deployments:
  - name: test_model
    entrypoint: modal_apps/modal_prefect_test.py:run_app
When do the pull, build, push steps happen here? I don't see any logs associated with these steps
1
k
build and push steps run when you do
prefect deploy
and pull steps run right before the start of a flow run
does your CI pipeline say
run: prefect deploy
exactly? You'll need to specify deployments by name like
prefect deploy -n test_model
or do
prefect deploy --all
to run the steps for all the deployments in your
prefect.yaml
c
Awesome, will try this ty
tyvm getting further now 🙂
k
🙌
c
Is there a way to pull the image that is created in build?
Or does the pull inject the code into the image or something? Still confused by the workflow
k
depends on how you want to do things, but one thing I'm not seeing in your deployment is the
work_pool
section
to tell prefect which image you want to use, you'll need to set
job_variables
which is a dictionary where on of the keys is
image
c
Copy code
pull:
  - prefect.deployments.steps.git_clone:
      repository: <https://github.com/essexlabs/arnold.git>
      branch: feature/prefect-modal-testing
      access_token: "{{ secrets.GITHUB_TOKEN }}"

build:
  - prefect_docker.deployments.steps.build_docker_image:
      requires: prefect-docker
      image_name: arnold/base
      tag: dev
      dockerfile: dockerfiles/Dockerfile.deps

push: # Prefect blocks add docker stuff
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker
      image_name: "{{ build-image.image_name }}"
      tag: "{{ build-image.tag }}"
      credentials: "{{ prefect.blocks.docker-registry-credentials.dev-registry }}"
      additional_tags: "{{ build-image.additional_tags }}"

deployments:
  - name: test_model
    entrypoint: modal_apps/test_modal_app.py:run_app
    work_pool:
      name: modal-pool
I have this now
(still need to setup the github credentials - access_token: "{{ prefect.blocks.secret.github-access-token }}" # Requires creation of a Secret block)
Planning to write out all these steps in a github gist or something to share with everyone 🙂 some of this stuff isn't super clear to me. Can send to modal folks too
k
okay so a couple things to go over here. similar to most CI processes, you'll need to add a step
id
so you can refer to the outputs of previous steps by name
Copy code
build:
  - prefect_docker.deployments.steps.build_docker_image:
      id: build-image  <-- now "{{ build-image.<output-name> }}" will work in later steps
      requires: prefect-docker
      image_name: arnold/base
      tag: dev
      dockerfile: dockerfiles/Dockerfile.deps
then on your deployment, you can access the combined
image:tag
string as "{{ build-image.image }}"
Copy code
work_pool:
  name: modal-pool
  job_variables:
    image: "{{ build-image.image }}"
job_variables get attached to your deployment, and override the defaults from your work pool
c
awesome ty
k
then modal will boot up your image as a container, and then prefect will clone your code into the container's current working directory, find your flow entrypoint, and execute it
c
perfect ty ty