https://prefect.io logo
Title
v

Vikram Iyer

06/30/2020, 7:11 PM
Hey All, I have run into a roadblock. 1. I have spun up the containers for apollo, ui, scheduler, postgres, hasura, graphql, and myapp using the docker-compose from HERE 2. I am able to register the flows and am able to see the registered flows at localhost:8080 3. To execute the flows, I have started a agent docker container in the same network as the other containers with command
prefect agent start docker --network prefect-server
I am still not able to execute the flows from the UI. Is there anything I am doing wrong?
z

Zachary Hughes

06/30/2020, 7:13 PM
Hi @Vikram Iyer, do you mind elaborating a bit on how you're unable to execute flows from the UI? Is the agent failing to pick up and submit your flows, or are the flows failing once they begin running?
v

Vikram Iyer

06/30/2020, 7:17 PM
The flows are in this state forever.
@Zachary Hughes
The agent run logs on cli.
z

Zachary Hughes

06/30/2020, 7:18 PM
That's super useful, thank you. Can you check to see if your flows have been registered with labels? The most common culprit in this sort of situation is a flow run/agent label mismatch.
v

Vikram Iyer

06/30/2020, 7:19 PM
ok, i just used the flow.register() without any labels. Should I add one and verify?
z

Zachary Hughes

06/30/2020, 7:21 PM
If you used the basic
flow.register()
with local storage, then your flow was likely auto-tagged with several labels specific to your machine. If you want to run your registered flows with local storage, I'd recommend spinning up a local agent. And if you'd rather work with a Docker agent, you'll be best off registering your flows with Docker storage.
v

Vikram Iyer

06/30/2020, 7:22 PM
Thanks for the immediate reply, @Zachary Hughes. How does one spin up a local agent?
z

Zachary Hughes

06/30/2020, 7:23 PM
v

Vikram Iyer

06/30/2020, 7:23 PM
Oh, you mean
prefect agent start
spins up a local agent and
prefect agent start docker
spins up a docker based agent and hence I might have to use the docker storage to access the registered flows. Am i getting it right?
z

Zachary Hughes

06/30/2020, 7:27 PM
Correct. Since local storage tends to be specific to one machine, flows registered with local storage are auto-labeled with information about that machine. Similarly, running a local agent on that same machine auto-detects flows running with that machine's labels. Since Docker storage is meant to be more portable, we don't do any sort of auto-labelling. So if you want to move your flow between machines or avoid labels altogether, registering your flow with Docker storage is a good way to go.
v

Vikram Iyer

06/30/2020, 7:38 PM
I'll try implementing your comments and let you know. Thanks for your help again @Zachary Hughes 🙂
z

Zachary Hughes

06/30/2020, 7:41 PM
Sure thing, any time!
v

Vikram Iyer

06/30/2020, 7:51 PM
Just to confirm once again, local agent can only be used to run the flows programatically right? like
flow.run_agent()
To execute from the UI, when using docker containers for other services, I will have to use Docker Storage. Sorry if this is redundant, the terms are quite confusing to be honest. @Zachary Hughes
z

Zachary Hughes

06/30/2020, 7:54 PM
No problem! One minor tweak to what you said: You can absolutely execute flows from the UI using local storage. In order to do so, you'd want to register your flow locally, then spin up a local agent. With that setup, you'd be able to run flows from the UI even with local storage.
v

Vikram Iyer

06/30/2020, 8:14 PM
Ohh! So I checked this example from the documentation.
from prefect import Flow
from prefect.environments.storage import Local

flow = Flow("local-flow", storage=Local())

flow.storage.build()
So, does the
flow.storage.build()
replace
flow.register()
or are we required to do both?
z

Zachary Hughes

06/30/2020, 8:19 PM
You'll still need to register your flow.
.build()
stores the flow as bytes, while
.register()
tells the API information about your flow.
l

Laura Lorenz (she/her)

07/31/2020, 3:20 PM
@Marvin archive “Why is my flow stuck in Scheduled state?”