Ok, I think I can finally explain the core of what...
# ask-community
m
Ok, I think I can finally explain the core of what I've been confused about. Setup: • Running a container on cloud run that uses prefect.serve() • Prefect.yaml has a build step for the container, and a push step that pushes it to artifact registry and starts a persistent service (with 1 instance if that matters) Problem: • If I just do those two steps and then send jobs from the UI, everything works fine • If I try to do any flavor of
prefect deploy
, it tells me I need to create a workpool. I'm not clear why, the documentation for static infrastructure seems to say they are optional> • If I go ahead and create a workpool, the workpool page in the UI says it isn't ready, that I need to run
prefect worker start
- but why would I need to do that, my container is already succesfully polling for jobs Am I just not supposed to be using prefect.yaml if I don't need a workpool? Is that message wrong? Am I configuring it wrong? Should I have a pasta or pizza? Here's my prefect.yaml - thank you so much for any help!! (This is in the state before running
prefect deploy --all
and having it want to add a workpool)
Copy code
name: ringmaster
prefect-version: 3.0.0

build:
- prefect.deployments.steps.run_shell_script:
    script: |
      docker build -t us-central1-docker.pkg.dev/michaeln-test-382306/prefect-images/ringmaster:latest .

push:
- prefect.deployments.steps.run_shell_script:
    script: |
      docker push us-central1-docker.pkg.dev/michaeln-test-382306/prefect-images/ringmaster:latest
      gcloud run deploy 'ringmaster' \
        --image 'us-central1-docker.pkg.dev/michaeln-test-382306/prefect-images/ringmaster:latest' \
        --region 'us-central1' \
        --platform 'managed' \
        --port '8080' \
        --min-instances '1' \
        --max-instances '1' \
        --cpu '1' \
        --memory '2048Mi' \
        --timeout '3600' 

pull:
- prefect.deployments.steps.set_working_directory:
    directory: /app

# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: ringmaster-short
  version:
  tags: []
  description:
  schedule: {}
  entrypoint: ringmaster.py:run_weekly_ringmaster
  concurrency_limit:
  parameters:
    short: true
    test_send: false
  enforce_parameter_schema: true
  schedules:
  - cron: 0 17 * * 1-4
    timezone: America/Los_Angeles
    day_or: true
    active: true
b
Hi Michael, personally, I'd pick pizza.
I can elaborate later on that if you really need me to ^
In the meantime though, let me do my best to help you out with your other question
Running the
.serve()
method does two main things: 1. Creates a deployment for the flow (you can see the deployment record in the UI). 2. Starts a long-running process that connects to your prefect server, watches for scheduled work related to the deployment mentioned above, and runs that scheduled work when necessary. If you're using
.serve()
you do not need a work pool or a
prefect.yaml
.
In this case, you don't need to run
prefect deploy
from the CLI either. Running that command is only necessary for creating a new
prefect.yaml
, or referencing an existing one whenever you need to deploy flows from one of your directories/repositories.
In essence, this is a tale of two approaches to deployments. If you want static infra, just a long running process that runs forever,
.serve()
is the way to go. Otherwise, if you're looking at dynamic infra that scales up and down with your workflow volume, the world of work pools, workers,
prefect.yaml
s is where you'd need to be.
m
Thank you, that's so incredibly helpful!
🚀 1
Goodbye prefect.yaml, hello pizza.
🍕 1
💯 1