Help needed / Question -- I have a local agent run...
# prefect-server
Help needed / Question -- I have a local agent running in a docker container that is properly picking up a flow i publish from another container but failing with the following error message. "Failed to load and execute Flow's environment: ValueError('Flow is not contained in this Storage')" The flow is assigned a UniversalRun (i tried a LocalRun with the same effect), a LocalExecutor, LocalStorage, and labels ABC and DEF. The Flow has labels ABC, DEF and is properly picked up by the agent.
Version is Prefect 4.16 using the prefect docker images.
Hey @Noah Guilbault! It is able to load the container but can’t find the Flow in the container? Could you show the Dockerfile and how you set the Storage?
The dockerfile mounts the prefect home directory as /prefect_home on all agents as well as on the container performing the client.register(flow, ...) call. the config.toml also specified the location.
I’m a bit confused. Why are you using
local agent
to pick up a flow from another container? I think
docker agent
might be better?
it's a distributed file system that is bind mounted where it needs to be accessed.
Gotcha. I think this is likely a relative path issue. I assume the Docker image running the agent has access to the bind mount. I think the way to go is to use LocalStorage (which is used by default). The issue is that there is a default location where the Flows are saved, and it will look for the flows in that location.
The agent will look for that default location and won't find the files so you need to put the flow files there. You need to use
but supply a path where the files would also exist for the agent.
The more common approach I've seen is Docker agent + LocalStorage where the files are stored relative to inside the Docker container.
I have an example of DockerRun + LocalStorage. Do you want to see it?
Does the place that you registered the flow from have access to the distributed file system?
yes -- seeing that would be great!
This is a recipe for DockerRun + LocalStorage
thanks a bunch!
just an update - i've gotten everyhting up and working reasonably well.
using local, and docker agents.
👍 1