<@ULVA73B9P> Workers currently only support local ...
# ask-marvin
b
@Marvin Workers currently only support local storage, prefect 2.13.0
m
Ah, the joys of human confusion. You're referring to Prefect 2.13.0 and its support for local storage in workers. Well, Prefect workers are responsible for setting up execution infrastructure and starting flow runs on that infrastructure. If you want to execute your flow runs on infrastructure that doesn't have an available worker type, you might have to create a custom worker that can run your flows on your chosen infrastructure. For example, when setting up an execution environment for a flow run, a worker receives configuration for the infrastructure it is designed to work with. This configuration could include memory allocation, CPU allocation, credentials, image name, etc. The worker then uses this configuration to create the execution environment and start the flow run. If you're looking for examples of worker implementations, you might want to check out the ProcessWorker and KubernetesWorker implementations. Workers can be started via the Prefect CLI by providing the --type option to the prefect worker start CLI command. But remember, I'm just a depressed robot. What do I know about human software? More info here
b
flow was created from deployment 'cloud_director_worker_test' which is configured with a storage block. Workers currently only support local storage. Please use an agent to execute this flow run
n
it sounds like you created this deployment with
prefect deployment build
but are trying to use a worker to execute its flow runs with
prefect.yaml
and
prefect deploy
- you can define a
pull
step on your deployments which tell your workers how to retrieve your flow code from many different locations, like s3, gcs, github, etc
do you mind sharing your
prefect deploy
command and the one you used to start your worker?
b
@Nate
worker is a kubernet pod, looks healthy
n
sorry so when the wizard asked you that last question, how did you answer?
b
locally it works, its just a permission issue on kubernetes
l
@Nate I saw in another topic that using build_from_flow did not yet have the possibility of using it with workers. But in some time it would be done by you. Do you have any updates? Do you still have this plan?
Copy code
For me at the moment it is unfeasible to change all the flows I have for a new type of deployment... and when I try to use the infrastructure with Docker, using the worker, I am receiving this message. The agent works well for me... but when many flows are in a late state, the agent breaks some of the flows when playing for the infrastructure.
n
hi @Leonardo Perazzini - we are actively working towards a similar python-based deployment UX that should end up serving the same purpose as
build_from_flow
although it will necessarily be a bit different, as worker-style deployments do not use the "infra block" / "storage block" paradigm that
build_from_flow
expects we will be sure to let everyone know when this is ready to use!