https://prefect.io logo
#prefect-community
Title
# prefect-community
m

matt_innerspace.io

01/07/2020, 9:05 PM
One simple thing i'm not grasping here - how do i deploy a flow that exists in a
hello_world.py
script? Lots of examples of what can happen inside the python script, but i'm missing how it would be deployed as a flow. I can see the flow runs inside a docker container, on an agent. I can see how the
prefect
cli tool can run a flow, but it seems the deployment and running are linked somehow? For comparison, i use openfaas python functions which also run in docker containers and are deployed wherever has room to run it, but there is an explicit
faas-cli push ...
to deploy to the cloud/cluster when you're ready.
z

Zachary Hughes

01/07/2020, 9:37 PM
Hi Matt, Using an example
hello_world.py
file containing a hello world flow as an example:
Copy code
from prefect import task, Flow

@task
def hello_world():
    print("Hello world!")


with Flow("Hello") as flow:
    hello = hello_world()
You can then import, run, and/or register it the same way you would as if it were in the same script:
Copy code
from hello_world import flow
flow.run()
flow.register(project_name=YOUR_PROJECT_NAME)
Does that help at all?
m

matt_innerspace.io

01/07/2020, 9:40 PM
ok, so the deployment happens within a python script... and that python code is pickled and stored in the cloud, then distributed to agents somehow.
z

Zachary Hughes

01/07/2020, 9:45 PM
Yep! Once your flow is registered with Cloud, it'll be available for you to create runs. Once a run is created, your agent will pick it up, presuming the labels on the agent and the flow match. If you'd like to test this out all in one step, you can run:
Copy code
flow.run_agent()
As described here: https://docs.prefect.io/cloud/tutorial/first.html#run-flow-with-prefect-cloud
j

Jeremiah

01/08/2020, 12:26 AM
With one caveat - the code itself is never pickled or sent to Prefect. We don’t (and can’t) receive code; it always remains on your infrastructure. What’s sent to us is metadata about the flow so that Cloud knows enough to be able to orchestrate the various tasks; but you’ll need to make sure your agent has access to the code (which is made easy by using one of Prefect Core’s
Environments
- local, docker, etc.)
z

Zachary Hughes

01/08/2020, 2:07 AM
^^ Excellent point.
m

matt_innerspace.io

01/08/2020, 3:42 PM
Environments
and
Storage
is where I'm getting tripped up - storage (local, s3, docker, etc) is where the code is deployed, which wasn't clear to me. I reworked the example above in a gist to deploy to my own docker repo, which still seems to fail... I'm missing something simple here - https://gist.github.com/qmnonic/921f5847af4bd7e5808a0eca61aeb6ed
c

Chris White

01/08/2020, 4:15 PM
Hi Matt, while this error message is very uninformative, I believe it’s occurring whenever your docker daemon attempts to push your image. If you are trying to push to docker hub, you might need to update your docker configuration to:
Copy code
Docker(registry_url="innerspace", image_name="a-repo-within-your-dockerhub")
every container registry type interacts with docker’s naming conventions differently
we’ll take a look at improving the quality of that message
m

matt_innerspace.io

01/08/2020, 6:07 PM
ok, that worked.. thanks!
👍 1
c

Chris White

01/08/2020, 6:54 PM
@Marvin archive “Building Docker Storage for DockerHub fails with error”