https://prefect.io logo
Powered by
Title
d

Deckel de Lange

06/30/2021, 11:59 AM
Hi all! Does anyone have experience following this guide to running a prefect server on GCP? https://medium.com/the-prefect-blog/prefect-server-101-deploying-to-google-cloud-platform-47354b16afe2 All has gone well until the last stage 
prefect server start
where I get the following error: 
ERROR: Couldn't connect to Docker daemon at <http+docker://localhost> - is it running?
any help would be appreciated thanks!
✅ 1
Full error logs: 
ERROR: Couldn't connect to Docker daemon at <http+docker://localhost> - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
Exception caught; killing services (press ctrl-C to force)
ERROR: Couldn't connect to Docker daemon at <http+docker://localhost> - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
Traceback (most recent call last):
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/prefect/cli/server.py", line 609, in start
    ["docker-compose", "pull"], cwd=compose_dir_path, env=env
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker-compose', 'pull']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/prefect-runner/prefect-env/bin/prefect", line 11, in <module>
    load_entry_point('prefect==0.14.22+28.g4218c2f73', 'console_scripts', 'prefect')()
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/prefect-runner/prefect-env/lib/python3.6/site-packages/prefect/cli/server.py", line 650, in start
    ["docker-compose", "down"], cwd=compose_dir_path, env=env
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.
g

Greg Roche

06/30/2021, 12:23 PM
I had a similar error when installing to an EC2 instance on AWS, for me it was due to the 
docker-compose
binary not being given execution permissions. Maybe try rerunning these commands to ensure docker-compose is installed and can be run: 
sudo curl -L "<https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname> -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
👍 1
:upvote: 1
d

Deckel de Lange

06/30/2021, 12:30 PM
Amazing thank you @Greg Roche, I believe permission were the problem yes 💯👏🎉
🎉 1
👍 1
i

Ile Lee

07/29/2021, 8:23 PM
@Greg Roche Hi. Is there a documentation somewhere online that goes through how to host your prefect server on aws? I am really struggling
g

Greg Roche

07/30/2021, 7:25 AM
I'm not aware of any documentation specifically for EC2, but Prefect did write a blog post about how to install it on GCP here: https://medium.com/the-prefect-blog/prefect-server-101-deploying-to-google-cloud-platform-47354b16afe2 . The first bit of the post is very specific to GCP but once you've set up your EC2 instance (we are using 
Ubuntu Server 18.04 LTS (HVM)
on a 
t2.medium
instance) the steps from "Installing dependencies" and onwards are basically identical.
i

Ile Lee

07/31/2021, 9:09 AM
@Greg Roche I've had a look at that and tried my best to do the same on AWS. There aren't many people who have done so, and since you have, could you correct me if any of my steps are wrong? Sorry I'm still quite new to this field. This is what I've done in the N.Virginia region (us-east-1): Step 1: Created VPC • with CIDR block 10.100.0.0/16 Step 2: Created Subnet • in us-east-1b • with CIDR block 10.100.1.0/24 • enabled auto assign public IPv4 address Step 3: Created Internet Gateway • then attached it to the VPC Step 4: Created Route Table • Added destination 0.0.0.0 to target the internet gateway • In subnet association, added the subnet to explicit subnet association Step 5: Created EC2 • Also created Security Group - added inbound rules TCP 8080 (for prefect UI), TCP 4200 (for graphql), TCP 22 (for SSH) to anyone (0.0.0.0/0) • Downloaded a new key pair Step 6: SSH into the EC2 instance • updated packages • installed docker, docker-compose, prefect (ran sudo service docker start, so docker ps works) Step 7: Created a directory in /home/ec2-user/.prefect Step 8 : Created a config.toml file in /home/ec2-user/.prefect with the following inside it 
[server]
       [server.ui]
       apollo_url = "<http://public-ip:4200/graphql>"
       graphql_url = "<http://public-ip:4200/graphql>"
From there: prefect backend server prefect server start The server starts and says you can visit in localhost:8080 but when I go to my-public-ip:8080, the prefect UI doesn't show I'm sorry for such a long explanation, I just can't figure out why I can't get it running :(
I am using Linux AMI 2
3 Views
#prefect-serverJoin Slack
Powered by