https://prefect.io logo
Title
n

Noah Guilbault

05/10/2021, 11:11 PM
Help needed / Question -- I have a local agent running in a docker container that is properly picking up a flow i publish from another container but failing with the following error message. "Failed to load and execute Flow's environment: ValueError('Flow is not contained in this Storage')" The flow is assigned a UniversalRun (i tried a LocalRun with the same effect), a LocalExecutor, LocalStorage, and labels ABC and DEF. The Flow has labels ABC, DEF and is properly picked up by the agent.
Version is Prefect 4.16 using the prefect docker images.
k

Kevin Kho

05/11/2021, 3:20 PM
Hey @Noah Guilbault! It is able to load the container but can’t find the Flow in the container? Could you show the Dockerfile and how you set the Storage?
n

Noah Guilbault

05/12/2021, 4:49 PM
The dockerfile mounts the prefect home directory as /prefect_home on all agents as well as on the container performing the client.register(flow, ...) call. the config.toml also specified the location.
k

Kevin Kho

05/12/2021, 5:00 PM
I’m a bit confused. Why are you using
local agent
to pick up a flow from another container? I think
docker agent
might be better?
n

Noah Guilbault

05/13/2021, 12:56 AM
it's a distributed file system that is bind mounted where it needs to be accessed.
k

Kevin Kho

05/13/2021, 12:59 AM
Gotcha. I think this is likely a relative path issue. I assume the Docker image running the agent has access to the bind mount. I think the way to go is to use LocalStorage (which is used by default). The issue is that there is a default location where the Flows are saved, and it will look for the flows in that location.
The agent will look for that default location and won't find the files so you need to put the flow files there. You need to use
LocalStorage
but supply a path where the files would also exist for the agent.
The more common approach I've seen is Docker agent + LocalStorage where the files are stored relative to inside the Docker container.
I have an example of DockerRun + LocalStorage. Do you want to see it?
Does the place that you registered the flow from have access to the distributed file system?
n

Noah Guilbault

05/13/2021, 2:01 PM
yes -- seeing that would be great!
k

Kevin Kho

05/13/2021, 2:05 PM
This is a recipe for DockerRun + LocalStorage
n

Noah Guilbault

05/24/2021, 5:49 PM
thanks a bunch!
just an update - i've gotten everyhting up and working reasonably well.
using local, and docker agents.
👍 1