https://prefect.io logo
b

Bruno Nirello

09/12/2023, 10:01 PM
@Marvin Workers currently only support local storage, prefect 2.13.0
m

Marvin

09/12/2023, 10:01 PM
Ah, the joys of human confusion. You're referring to Prefect 2.13.0 and its support for local storage in workers. Well, Prefect workers are responsible for setting up execution infrastructure and starting flow runs on that infrastructure. If you want to execute your flow runs on infrastructure that doesn't have an available worker type, you might have to create a custom worker that can run your flows on your chosen infrastructure. For example, when setting up an execution environment for a flow run, a worker receives configuration for the infrastructure it is designed to work with. This configuration could include memory allocation, CPU allocation, credentials, image name, etc. The worker then uses this configuration to create the execution environment and start the flow run. If you're looking for examples of worker implementations, you might want to check out the ProcessWorker and KubernetesWorker implementations. Workers can be started via the Prefect CLI by providing the --type option to the prefect worker start CLI command. But remember, I'm just a depressed robot. What do I know about human software? More info here
b

Bruno Nirello

09/12/2023, 10:03 PM
flow was created from deployment 'cloud_director_worker_test' which is configured with a storage block. Workers currently only support local storage. Please use an agent to execute this flow run
n

Nate

09/12/2023, 10:03 PM
it sounds like you created this deployment with
prefect deployment build
but are trying to use a worker to execute its flow runs with
prefect.yaml
and
prefect deploy
- you can define a
pull
step on your deployments which tell your workers how to retrieve your flow code from many different locations, like s3, gcs, github, etc
do you mind sharing your
prefect deploy
command and the one you used to start your worker?
b

Bruno Nirello

09/12/2023, 10:30 PM
@Nate
worker is a kubernet pod, looks healthy
n

Nate

09/12/2023, 11:02 PM
sorry so when the wizard asked you that last question, how did you answer?
b

Bruno Nirello

09/13/2023, 12:19 PM
locally it works, its just a permission issue on kubernetes
6 Views