Michael Natkin
11/21/2024, 6:52 PMprefect deploy
, it tells me I need to create a workpool. I'm not clear why, the documentation for static infrastructure seems to say they are optional>
• If I go ahead and create a workpool, the workpool page in the UI says it isn't ready, that I need to run prefect worker start
- but why would I need to do that, my container is already succesfully polling for jobs
Am I just not supposed to be using prefect.yaml if I don't need a workpool? Is that message wrong? Am I configuring it wrong? Should I have a pasta or pizza?
Here's my prefect.yaml - thank you so much for any help!! (This is in the state before running prefect deploy --all
and having it want to add a workpool)
name: ringmaster
prefect-version: 3.0.0
build:
- prefect.deployments.steps.run_shell_script:
script: |
docker build -t us-central1-docker.pkg.dev/michaeln-test-382306/prefect-images/ringmaster:latest .
push:
- prefect.deployments.steps.run_shell_script:
script: |
docker push us-central1-docker.pkg.dev/michaeln-test-382306/prefect-images/ringmaster:latest
gcloud run deploy 'ringmaster' \
--image 'us-central1-docker.pkg.dev/michaeln-test-382306/prefect-images/ringmaster:latest' \
--region 'us-central1' \
--platform 'managed' \
--port '8080' \
--min-instances '1' \
--max-instances '1' \
--cpu '1' \
--memory '2048Mi' \
--timeout '3600'
pull:
- prefect.deployments.steps.set_working_directory:
directory: /app
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: ringmaster-short
version:
tags: []
description:
schedule: {}
entrypoint: ringmaster.py:run_weekly_ringmaster
concurrency_limit:
parameters:
short: true
test_send: false
enforce_parameter_schema: true
schedules:
- cron: 0 17 * * 1-4
timezone: America/Los_Angeles
day_or: true
active: true
Bianca Hoch
11/21/2024, 9:37 PMBianca Hoch
11/21/2024, 9:37 PMBianca Hoch
11/21/2024, 9:38 PMBianca Hoch
11/21/2024, 9:46 PM.serve()
method does two main things:
1. Creates a deployment for the flow (you can see the deployment record in the UI).
2. Starts a long-running process that connects to your prefect server, watches for scheduled work related to the deployment mentioned above, and runs that scheduled work when necessary.
If you're using .serve()
you do not need a work pool or a prefect.yaml
.Bianca Hoch
11/21/2024, 9:53 PMprefect deploy
from the CLI either. Running that command is only necessary for creating a new prefect.yaml
, or referencing an existing one whenever you need to deploy flows from one of your directories/repositories.Bianca Hoch
11/21/2024, 10:02 PM.serve()
is the way to go. Otherwise, if you're looking at dynamic infra that scales up and down with your workflow volume, the world of work pools, workers, prefect.yaml
s is where you'd need to be.Michael Natkin
11/21/2024, 11:17 PMMichael Natkin
11/21/2024, 11:17 PMMichael Natkin
11/21/2024, 11:19 PM