Hey All, I have run into a roadblock. 1. I have spun up the containers for apollo, ui, scheduler, postgres, hasura, graphql, and myapp using the docker-compose from HERE 2. I am able to register the flows and am able to see the registered flows at localhost:8080 3. To execute the flows, I have started a agent docker container in the same network as the other containers with command

prefect agent start docker --network prefect-server

I am still not able to execute the flows from the UI. Is there anything I am doing wrong?

Hi @Vikram Iyer, do you mind elaborating a bit on how you're unable to execute flows from the UI? Is the agent failing to pick up and submit your flows, or are the flows failing once they begin running?

The flows are in this state forever.

That's super useful, thank you. Can you check to see if your flows have been registered with labels? The most common culprit in this sort of situation is a flow run/agent label mismatch.

ok, i just used the flow.register() without any labels. Should I add one and verify?

If you used the basic

flow.register()

with local storage, then your flow was likely auto-tagged with several labels specific to your machine. If you want to run your registered flows with local storage, I'd recommend spinning up a local agent. And if you'd rather work with a Docker agent, you'll be best off registering your flows with Docker storage.

Thanks for the immediate reply, @Zachary Hughes. How does one spin up a local agent?

This should get you up and running! https://docs.prefect.io/orchestration/agents/local.html#local-agent

Oh, you mean

prefect agent start

spins up a local agent and

prefect agent start docker

spins up a docker based agent and hence I might have to use the docker storage to access the registered flows. Am i getting it right?

Correct. Since local storage tends to be specific to one machine, flows registered with local storage are auto-labeled with information about that machine. Similarly, running a local agent on that same machine auto-detects flows running with that machine's labels. Since Docker storage is meant to be more portable, we don't do any sort of auto-labelling. So if you want to move your flow between machines or avoid labels altogether, registering your flow with Docker storage is a good way to go.

I'll try implementing your comments and let you know. Thanks for your help again @Zachary Hughes 🙂

Sure thing, any time!

Just to confirm once again, local agent can only be used to run the flows programatically right? like

flow.run_agent()

To execute from the UI, when using docker containers for other services, I will have to use Docker Storage. Sorry if this is redundant, the terms are quite confusing to be honest. @Zachary Hughes

No problem! One minor tweak to what you said: You can absolutely execute flows from the UI using local storage. In order to do so, you'd want to register your flow locally, then spin up a local agent. With that setup, you'd be able to run flows from the UI even with local storage.

Ohh! So I checked this example from the documentation.

from prefect import Flow
from prefect.environments.storage import Local

flow = Flow("local-flow", storage=Local())

flow.storage.build()

So, does the

flow.storage.build()

replace

flow.register()

or are we required to do both?

You'll still need to register your flow.

.build()

stores the flow as bytes, while

.register()

tells the API information about your flow.

@Marvin archive “Why is my flow stuck in Scheduled state?”