Javier Ochoa
08/30/2023, 6:07 PMTraceback (most recent call last):
File "/home/javier/git/nzero/usg-prefect-2/.venv/lib/python3.9/site-packages/prefect/workers/base.py", line 784, in _submit_run
await self._check_flow_run(flow_run)
File "/home/javier/git/nzero/usg-prefect-2/.venv/lib/python3.9/site-packages/prefect/workers/base.py", line 770, in _check_flow_run
raise ValueError(
ValueError: Flow run UUID('53a4f0fc-1802-4986-8a72-a15cbd1f4674') was created from deployment 'my-worker' which is configured with a storage block. Workers currently only support local storage. Please use an agent to execute this flow run.
Nate
08/30/2023, 6:09 PMprefect deploy
since switching to prefect worker start
?
the deployment you create with prefect deploy
will be a little bit different from the one you get with prefect deployment build
Javier Ochoa
08/30/2023, 6:13 PMstorage = GitHub.load(settings.DOMAIN_ENV_HOST_ID)
work_pool_name = (
f"{settings.DOMAIN}-{settings.ENVIRONMENT}-agent-{settings.LOCALHOST_ID}"
)
Deployment.build_from_flow(
flow=flow_function,
name=settings.DOMAIN_ENV_HOST_ID,
tags=[settings.LOCALHOST_ID],
storage=storage,
work_pool_name=work_pool_name,
ignore_file=f"{settings.PROJ_ROOT_PATH}/.prefectignore",
apply=True,
)
Nate
08/30/2023, 6:22 PMprefect deployment build
& Deployment.build_from_flow()
) was designed for agents.
workers are designed to pick up and submit deployments created via `prefect.yaml` and `prefect deploy` . though we plan to have a pythonic interface for worker / workpool deployments soon, we dont have one yet.
so if you'd like to define your deployments in python for now, I might just stay on agents until we have a python interface released that's designed with workers in mind. you could also test out prefect.yaml
and prefect deploy
with any new flows you're writing to see how it goesJavier Ochoa
08/30/2023, 6:27 PMprefect.yaml
file
So ok, I will keep using agents until the next version. Thanks!Nate
08/30/2023, 6:30 PMmanage variables, env vars, string replacementthis all should be possible in
prefect.yaml
- so when the time comes feel free to ask about any troubles you have with thatJavier Ochoa
08/30/2023, 7:00 PMNate
08/30/2023, 8:58 PMJavier Ochoa
08/31/2023, 5:38 PM# Welcome to your prefect.yaml file! You can use this file for storing and managing
# configuration for deploying your flows. We recommend committing this file to source
# control along with your flow code.
# Generic metadata about this project
name: usg-prefect-2
prefect-version: 2.11.5
# build section allows you to manage and build docker images
build: null
# push section allows you to manage if and how this project is uploaded to remote locations
push: null
# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.git_clone:
repository: git@github.com:placeholder/placeholder.git
branch: dev
credentials: "{{ prefect.blocks.github-credentials.placeholder }}"
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: water-flow
version: null
tags: []
description: null
schedule: {}
flow_name: null
entrypoint: flows/water-flow.py:main
parameters: {}
work_pool:
name: "{{ $LOCALHOST_ID }}"
work_queue_name: default
job_variables: {}
This deployment configuration references work pool '{{ $LOCALHOST_ID }}' which does not exist. This means no worker will be able to pick up its runs. You can create a work pool in
the Prefect UI.