https://prefect.io logo
Title
n

Nazeer Hussain

06/11/2020, 6:04 PM
Hello All, I am trying to run Prefect on Docker, and I am getting the following error : docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"-p\": executable file not found in $PATH": unknown. , Can someone help please
👀 1
d

Dylan

06/11/2020, 6:06 PM
Hi @Nazeer Hussain! Can you tell me a little bit more about the steps you took that lead up to this error including: 1. Are you trying to spin up Prefect Server 2. Are you attempting to run a flow? 3. Is
prefect
available in your path?
n

Nazeer Hussain

06/11/2020, 6:06 PM
Hi Dylan
I have just pulled the Prefect image from Dockerhub : https://hub.docker.com/r/prefecthq/prefect
and trying to run it first, am i missing something, please help
Yes, You are right, first I need to run a Prefect server in Docker
I want to run a prefect flow using the Docker , unable to find the right documentation
d

Dylan

06/11/2020, 6:08 PM
First, let’s make sure you have Docker Compose installed
n

Nazeer Hussain

06/11/2020, 6:09 PM
docker-compose version 1.25.5, build 8a1c60f6
Here’s the relevant doc
prefect backend server
prefect server start
n

Nazeer Hussain

06/11/2020, 6:10 PM
ok Should I install prefect manually first , as described in https://docs.prefect.io/core/getting_started/installation.html
d

Dylan

06/11/2020, 6:11 PM
Can you run
prefect version
for me?
n

Nazeer Hussain

06/11/2020, 6:11 PM
let me check
d

Dylan

06/11/2020, 6:11 PM
And yes, you’ll need to install Prefect into your current python environment
n

Nazeer Hussain

06/11/2020, 6:11 PM
-bash: prefect: command not found
d

Dylan

06/11/2020, 6:12 PM
Okay! Looks like
prefect
isn’t available in your $PATH
How are you managing your python dependencies?
pip
conda
etc
n

Nazeer Hussain

06/11/2020, 6:12 PM
ok one moment plz, installation in progress
prefect version 0.11.5
done
d

Dylan

06/11/2020, 6:13 PM
👍
Now try:
prefect backend server
prefect server start
n

Nazeer Hussain

06/11/2020, 6:13 PM
Sure, Thanks for helping, let me try that
prefect backend server Backend switched to server
d

Dylan

06/11/2020, 6:13 PM
Of course! That’s what we’re here for 😄
n

Nazeer Hussain

06/11/2020, 6:13 PM
🙂
My end goal is to run Prefect on AWS ECS
so wanted to try the docker way first
You are awesome @Dylan
:upvote: 1
d

Dylan

06/11/2020, 6:20 PM
Anytime!
😄
n

Nazeer Hussain

06/11/2020, 6:22 PM
how can i run a sample flow plz
Should be a great place to start!
n

Nazeer Hussain

06/11/2020, 6:24 PM
let me check
OK i create the test.py using the code :
import prefect
from prefect import task, Flow

@task
def hello_task():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>("Hello, Cloud!")

flow = Flow("hello-flow", tasks=[hello_task])
how do i run it please
d

Dylan

06/11/2020, 6:27 PM
Make sure you follow all the steps in https://docs.prefect.io/orchestration/tutorial/first.html
Including registering the Flow with your Prefect Server backend
n

Nazeer Hussain

06/11/2020, 6:27 PM
ok let me check again
d

Dylan

06/11/2020, 6:27 PM
But after that, you’ll be able to run the flow in a few ways outlined in the bottom of the doc
But the simplest should be by navigating to the Flow’s page in the Prefect Server UI and pressing the
Run
button
n

Nazeer Hussain

06/11/2020, 6:36 PM
Still not clear how to register a flow
😞
d

Dylan

06/11/2020, 6:37 PM
You’ll need to run the flow you registered in a python interpreter like
ipython
and then call
.register()
on the
flow
object
n

Nazeer Hussain

06/11/2020, 6:37 PM
ok let me try
d

Dylan

06/11/2020, 6:37 PM
What I typically do is: 1. write my flow 2. load the python file with
ipython -i flow.py
n

Nazeer Hussain

06/11/2020, 6:38 PM
ok..
d

Dylan

06/11/2020, 6:38 PM
3. run the flow with
flow.run()
and if that works 4. Register my flow with
flow.register()
all in the same ipython session
Prefect is designed so that you can run your flow on your laptop using only python
for testing / experimenting
n

Nazeer Hussain

06/11/2020, 6:39 PM
ok.. installing ipython using brew
d

Dylan

06/11/2020, 6:39 PM
Then when you’re ready, you register the flow with Cloud/Server and schedule it to run using a schedule or by pressing the
Run
button in the UI
n

Nazeer Hussain

06/11/2020, 6:40 PM
ok.. i will try
but i was unable to deploy the Prefect on AWS ECS.. do you have any reference link please
My task is to deploy it on the cloud using ECS< the dev team needs to use it for setup
ipython -i test.py  Python 3.8.3 (default, May 27 2020, 20:53:50)  Type 'copyright', 'credits' or 'license' for more information IPython 7.15.0 -- An enhanced Interactive Python. Type '?' for help. --------------------------------------------------------------------------- ModuleNotFoundError            Traceback (most recent call last) ~/prefect/test.py in <module> ----> 1 import prefect    2 from prefect import task, Flow    3     4 @task    5 def hello_task(): ModuleNotFoundError: No module named 'prefect'
d

Dylan

06/11/2020, 6:45 PM
Hi Nazeer, we don’t have a doc specifically for ECS
n

Nazeer Hussain

06/11/2020, 6:45 PM
oh ok
d

Dylan

06/11/2020, 6:45 PM
Here’s a blog post you can use as a guideline even though it’s GCP: https://medium.com/the-prefect-blog/prefect-server-101-deploying-to-google-cloud-platform-47354b16afe2
👍 1
If you need to do some quick experimentation with Prefect before you dive in to creating new infrastructure, Prefect Cloud has a free tier
n

Nazeer Hussain

06/11/2020, 6:46 PM
I will check it out
Sure will check Prefect cloud as well
d

Dylan

06/11/2020, 6:46 PM
To sign up, head to https://cloud.prefect.io
thanks to Cloud’s Hybrid Model, we never see any of your flow code or data
That all stays on your infrastructure
n

Nazeer Hussain

06/11/2020, 6:47 PM
ok
Sounds Great
ipython -i test.py  Python 3.8.3 (default, May 27 2020, 20:53:50)  Type 'copyright', 'credits' or 'license' for more information IPython 7.15.0 -- An enhanced Interactive Python. Type '?' for help. --------------------------------------------------------------------------- ModuleNotFoundError            Traceback (most recent call last) ~/prefect/test.py in <module> ----> 1 import prefect    2 from prefect import task, Flow    3     4 @task    5 def hello_task(): ModuleNotFoundError: No module named 'prefect'
what could be wrong
my prefect server is running
d

Dylan

06/11/2020, 6:49 PM
Python dependency management is tricky
Are you using pip or conda?
n

Nazeer Hussain

06/11/2020, 6:52 PM
pip3
d

Dylan

06/11/2020, 6:52 PM
Since you installed
ipython
with homebrew, it may not exist in the same python environment as
pip3
try
brew uninstall ipython
pip3 install ipython
n

Nazeer Hussain

06/11/2020, 6:53 PM
ok let me do that
d

Dylan

06/11/2020, 6:53 PM
And I’d recommend checking out Anaconda or pipenv to create separate python environments
n

Nazeer Hussain

06/11/2020, 6:53 PM
OK, thank you
d

Dylan

06/11/2020, 6:54 PM
(personally I use anaconda since it also works for non-python dependencies like C libraries and whatnot)
n

Nazeer Hussain

06/11/2020, 6:54 PM
I am not a python Expert, but an Architect, will still give it a try 🙂
Thanks Dylan
it finally worked
d

Dylan

06/11/2020, 7:22 PM
🎉
👍 1
Awesome!
👍 1