https://prefect.io logo
m

Mauricio Duarte Fiallos

07/07/2023, 5:44 PM
I created an ec2 instance in aws to install prefect and followed the installation process, and then wen to the host guide to initialize the service with prefect server start, but prefect command is not found, which means is not installed, I installed docker and sqlite, python3 is already installed by default, but I installed pip. What's the problem, am I doing something grown ?
j

jpuris

07/09/2023, 8:19 PM
Did you follow some tutorial or guide? If so, please share the steps. Alternatively, perhaps you installed prefect in a virtual environment? If so, then prefect must be run via that same venv. i.e.
Copy code
/path/to/my/venv/bin/prefect agent start
m

Mauricio Duarte Fiallos

07/09/2023, 8:27 PM
I followed the installation guide from the official site and I tried it at my local and with an ec2 instance, when I try to start the service doesn't recognize prefect as a command
j

jpuris

07/09/2023, 8:33 PM
Could you please a provide link to the official docs you've followed? 🤔
c

Christopher Boyd

07/09/2023, 8:46 PM
It’s probably just not in your path
Calling a binary like prefect works if it’s in your path . You can check to see what your path is via ‘echo $PATH’ . My guess is that where it was installed to by default doesn’t exist in your default path - You can check to see where it installed with “whereis prefect” and “locate prefect” and failing those , just a find like “find / -name ‘prefect’ -type f”
m

Mauricio Duarte Fiallos

07/09/2023, 8:55 PM
sure, here is the link https://docs.prefect.io/2.10.18/getting-started/installation/ and also screenshots from the documentation and my ec2 instance
@Christopher Boyd this is the output I get
c

Christopher Boyd

07/09/2023, 8:58 PM
It doesn’t look like it’s installed
Try ‘which prefect ‘
m

Mauricio Duarte Fiallos

07/09/2023, 9:01 PM
would this be the path you talking about?
j

jpuris

07/09/2023, 9:02 PM
I don't know about AWS Ubuntu image, but one from official docker hub (
:22.04
) installs prefect just fine out of the box 🤷 spin up the container
Copy code
docker run --rm -it --name prefect-ubuntu ubuntu:22.04
install
Copy code
apt update
apt install python3 python3-pip
pip install -U prefect
prefect is in path
Copy code
# which prefect
/usr/local/bin/prefect
I'm guessing it installs it under
.local/bin
because that is how AWS sets it up for non root users..
c

Christopher Boyd

07/09/2023, 9:04 PM
Yes , prefect was installed to .local/bin for that user . You can call the full path as it’s shown there, or update your path to include that directory , or move it
Or install it to another path
j

jpuris

07/09/2023, 9:05 PM
If you are fine with it installed under Ubuntu user like that, then you can run it with
Copy code
$ ~/.local/bin/prefect version
🤷
m

Mauricio Duarte Fiallos

07/09/2023, 9:06 PM
I think is kind of better to built my own docker image just following the steps @jpuris posted, is that right @Christopher Boyd, or would it be better to use an instance?
j

jpuris

07/09/2023, 9:08 PM
Please don't take my example running Ubuntu in container 😄 I only did it to try to reproduce EC2 environment. There is a prefect docker image you can use. Take a look at https://hub.docker.com/r/prefecthq/prefect
c

Christopher Boyd

07/09/2023, 9:09 PM
To add - it’s up to you . To me this is just a simple $PATH issue , nothing is broken
You can fix it by just exporting - export PATH=$PATH:/home/ubuntu/.local/bin
Docker is nice , as well kubernetes but a lot of that will be very dependent on your workload , your access, really how you intend to use it . If this is just for POC, I’d start with this first , because docker with prefect server will introduce some additional nuances like setting your api url, pointing to the docker internal host address and other things that might be easier getting right on a host first
m

Mauricio Duarte Fiallos

07/09/2023, 9:15 PM
in fact its a POC, I tried to run the docker image but didn't understand if ot was ok just by
docker run -d -p 4200:4200 prefecthq/prefect
then I was reading and still not sure about the DB (which I already installed), but the documentation says its configured when prefect is installed, how would I know the space necessary for the DB, should this DB run separately, like and RDS? would this be a justifiable cost or is better to run it in the host?
j

jpuris

07/09/2023, 9:22 PM
I'm not sure.. We rely entirely on prefect cloud for the server. Tho, if I am not mistaken, it by default runs local sqlite db and if you want to use a proper db, one would need to point it to a running postgresql db cluster edit: RDS would be a viable option in production setup. For POC you can roll your own local postgres or container, if you want to externalise the db
c

Christopher Boyd

07/09/2023, 9:23 PM
If you installed (or used an image) it comes with SQLite as jpuris mentioned. You can point to a Postgres instance if you want , following the prefect server docs - sizing and config is up to you
You don’t need to do anything else to make it work by default, just prefect server start and it will be backed by default using the SQLite instance
m

Mauricio Duarte Fiallos

07/09/2023, 9:26 PM
well this is nice, now a new issue arise jajaja
c

Christopher Boyd

07/09/2023, 9:26 PM
That’s an unfortunate bug on start, it should be fine
👍🏻 1
You can run your flows against it now , it’ll work
m

Mauricio Duarte Fiallos

07/09/2023, 9:27 PM
ok
I would have to ask, in terms of cost (just infra), do you think is better to run your own or would it be more cost effective to use the prefect cloud solution, I know for a fact that my boss will tell me that I'm the reason for him not to pay a subscription jajajajaja
c

Christopher Boyd

07/09/2023, 9:33 PM
Personally , I hear this question a lot - I can direct you to our product advocates who can give a much better description of value than I can but I’ll say that for the money , Cloud is a very good proposition . You absolutely can roll your own , but you’ll need to consider: Support Maintainability Uptime Patching Database maintenance Cost of both the infra to setup and run that The development / time cost to set it up and configure it
m

Mauricio Duarte Fiallos

07/09/2023, 9:33 PM
image.png
c

Christopher Boyd

07/09/2023, 9:34 PM
I think what we tend to see is some users setting up a dev environment with prefect server just to test and dev , but then prefect cloud for work
I think if you have questions / concerns about the value and whether it’s worth it , I’d love to direct you to @Sahil Rangwala and @Bianca Hoch - they can help answer a lot of these questions and determine what’s good for your use case
m

Mauricio Duarte Fiallos

07/09/2023, 9:36 PM
besides maintainability, support and stress free is there any limitation between the hosted version vs cloud? Are they the same but obviously cloud would be hassle free or they have different features?
c

Christopher Boyd

07/09/2023, 9:37 PM
They’d be able to help answer and demonstrate better but yes there are some differences here - https://docs.prefect.io/2.10.18/host/#differences-between-a-prefect-server-and-prefect-cloud
m

Mauricio Duarte Fiallos

07/09/2023, 9:39 PM
thanks man, you rock, don't forget to mention @jpuris for his contribution and advice
gracias totales