https://prefect.io
Join Slack
is there a way to add a label to a flow through the UI?
# prefect-ui
t
is there a way to add a label to a flow through the UI?
k
Try clicking the plus
t
Thanks, found it after i asked 😆 but now i’m facing two other problems: • flows i create using the CLI (or through python) on the EC2 machine end up getting the local IP as a label (even when i didn’t select that label), and then it tells me that no matching agents have that label… when i created the agent i just gave a single label (
poc
) and i try to give the same one to the flow i register but it also gets the host label… • even after i manually remove this label manually through the GUI, and try to run the flow - i get an error i don’t understand:
the run-config is a `UniversalRun`…
this is my entire flow:
Copy code
from prefect import task, Flow
from prefect.backend.artifacts import create_markdown_artifact
import requests


@task
def fetch():
    res = requests.get('<https://xkcd.com/454/info.0.json>')
    j = res.json()
    print(j)
    mrkdwn = "# My XKCD\n\n ![" + j['alt']+ "](" + j['img'] + ")"
    create_markdown_artifact(mrkdwn)


with Flow('xkcd') as flow:
    fetch()
flow was registered with:
Copy code
prefect register --project poc -p xkcd.py -l poc
agent was started with:
Copy code
prefect agent docker  start --key "XXX" -l poc
i’m guessing i need either a remote storage for the flow or… ?
k
You can turn off the default label by passing 
add_default_label=False
to your Storage. The default is local
t
ok that would solve the first bullet, what about the 2nd?
k
The default storage is local storage so Prefect serializes your Flow and saved it under the 
.prefect
folder by default. When the agent picks up the flow run, it goes to this place to fetch the flow and run it.
So if you register with local storage on machine A and run a flow on machine B that doesn’t have the file, you will get an error like this. If you need a Flow across machines, you need a remote storage
t
it is in fact the same machine (same EC2), but the agent is docker
k
Ah in this case, it’s being looked for inside the container, but it’s on your local.
This will work if the same file exists in the container at that specific location
t
so… how would i generally go about pushing flows into the agent docker?
(i’m asking more for sport/poc, obviously we’ll pick something more viable like s3 or github or whatever in production)
ah nvm i found it
k
That will work, but you can also make your own container and copy files is (if you have a lot of custom modules also). You can also use Docker storage and DockerRun to package that flow in a container
t
isn’t 
DockerRun
the default for docker agents?
k
No UniversalRun is the default which can be picked up by all agents but not really configured
t
ok, i meant - in practice, what will the 
DockerAgent
do when it gets a 
UniversalRun
flow to run?
wouldn’t it run it in a 
Docker
?
k
It will use the base prefecthq/prefect image yep
t
my problem with the 
Docker Storage
is that it expects a registry (so it becomes no better than using S3 which would now involve permissions etc. which im trying to avoid for the time being) Whoops nvm i see that’s optional
can 
Docker Agents
run flows who are configured with the 
Module Storage
? not sure what the module storage means (and it didn’t work when i tried, anyway) i guess it needs to be available in the docker image
k
Yes Docker Storage doesn’t need a registry. It can just be on local. Agent and RunConfigs are pairs but ALL Storage + RunConfig combinations can be used. Some just make less sense but they are all compatible. Yes the Module needs to be installed in the image.
t
wooo
k
Nice!
t
ok, one thing i’m still not clear on: if i configure my flow with a docker storage, does that mean it gets its own image (built locally, in this case)?
if i had a 2nd flow would that be a 2nd image?
k
Yes that’s right but if you want multiple flows in 1 image, you can follow this
t
ahhh i see, ya i remember running into this article before 😆 your docs are very exhaustive but there’s just so much of them
k
funny you say that. more commonly we hear not enough stuff is documented 😛
t
dunno i think i found every answer documented so far, i just sometimes struggle to get there myself through navigation or search (also, some questions would kind of require you to know where to look) i would consider having some FAQ format page in addition - plus maybe some kind of question-bot-thingie that can find you articles based on a human-readable question….. otherwise - you become the bot yourself 😆
k
I am working on our FAQ. I hope to get something out in early Jan
💯 1
t
the multi-flow thing worked btw
k
nice!
t
thx for the help, much appreciated 🙏
14 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by