https://prefect.io logo
Join Slack
Powered by
Do you know where this error comes from?
# prefect-community
r
Do you know where this error comes from?
j
Hey @Rafal this looks to be something related to your local running version of Docker 🤔 From what I found online it appears to be an issue that is specific to Ubuntu. Now sure if that’s what you’re running or not: https://stackoverflow.com/questions/45033688/invalid-ip-address-in-add-host-failed-to-program-filter-chain-iptables-fa
They fixed it by running:
Copy code
sudo ufw disable
sudo systemctl restart docker
Ah I also just remembered where I had seen this before: https://github.com/PrefectHQ/prefect/issues/2351 Are you by any chance running Docker in rootless mode?
r
Hmm, I checked and still the same. In corporate env you have different settings regarding VPN etc. All docker images are running perfectly fine, excluding prefect. Any more ideas?
j
Checked and it isn’t in rootless mode?
y
Sorry for chime in, just want to confirm that the rootless docker will give the add-host error when try to spin up the prefect server. My working env is Ubuntu 18.04, Docker 19.03 in rootless mode.
p
I am having the same issue on Ubuntu 18 running on WSL. I tried the suggestion from Josh above and that didn't work. Does anyone know how to make docker run in normal mode on WSL 2 ? I don't remember choosing a rootless option when I installed it.
The other question is what would it take to make this prefect setup run on rootless mode?
j
Interesting 🤔 I wonder if we can isolate this somehow. Let me see if I can make a minimal docker-compose file that only uses the postgres container and the network. I think it’s due to the network somehow
If you save this snippet as 
docker-compose.yml
and then run 
docker-compose up
(in the same directory) what happens?
Copy code
version: "3.7"

services:
  postgres:
    image: "postgres:11"
    ports:
      - "${POSTGRES_HOST_PORT:-5432}:5432"
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: test-password
      POSTGRES_DB: prefect_server
    networks:
      - prefect-server
    restart: "always"
    command:
      - "postgres"
      - "-c"
      - "max_connections=150"

networks:
  prefect-server:
    name: prefect-server
p
It seems to be working ok.
Copy code
pedro@DESKTOP-9OKD69C:.../Users/pedro/scratch$ docker-compose up
Creating network "prefect-server" with the default driver
Creating scratch_postgres_1 ... done
Attaching to scratch_postgres_1
postgres_1  | The files belonging to this database system will be owned by user "postgres".
postgres_1  | This user must also own the server process.
postgres_1  |
postgres_1  | The database cluster will be initialized with locale "en_US.utf8".
postgres_1  | The default database encoding has accordingly been set to "UTF8".
postgres_1  | The default text search configuration will be set to "english".
postgres_1  |
postgres_1  | Data page checksums are disabled.
postgres_1  |
postgres_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1  | creating subdirectories ... ok
postgres_1  | selecting default max_connections ... 100
postgres_1  | selecting default shared_buffers ... 128MB
postgres_1  | selecting default timezone ... Etc/UTC
postgres_1  | selecting dynamic shared memory implementation ... posix
postgres_1  | creating configuration files ... ok
postgres_1  | running bootstrap script ... ok
postgres_1  | performing post-bootstrap initialization ... ok
postgres_1  | syncing data to disk ... ok
postgres_1  |
postgres_1  | Success. You can now start the database server using:
postgres_1  |
postgres_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1  |
postgres_1  |
postgres_1  | WARNING: enabling "trust" authentication for local connections
postgres_1  | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1  | --auth-local and --auth-host, the next time you run initdb.
postgres_1  | waiting for server to start....2020-05-29 17:02:22.240 UTC [46] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2020-05-29 17:02:22.263 UTC [47] LOG:  database system was shut down at 2020-05-29 17:02:22 UTC
postgres_1  | 2020-05-29 17:02:22.272 UTC [46] LOG:  database system is ready to accept connections
postgres_1  |  done
postgres_1  | server started
postgres_1  | CREATE DATABASE
postgres_1  |
postgres_1  |
postgres_1  | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1  |
postgres_1  | 2020-05-29 17:02:22.691 UTC [46] LOG:  received fast shutdown request
postgres_1  | waiting for server to shut down....2020-05-29 17:02:22.696 UTC [46] LOG:  aborting any active transactions
postgres_1  | 2020-05-29 17:02:22.699 UTC [46] LOG:  background worker "logical replication launcher" (PID 53) exited with exit code 1
postgres_1  | 2020-05-29 17:02:22.699 UTC [48] LOG:  shutting down
postgres_1  | 2020-05-29 17:02:22.725 UTC [46] LOG:  database system is shut down
postgres_1  |  done
postgres_1  | server stopped
postgres_1  |
postgres_1  | PostgreSQL init process complete; ready for start up.
postgres_1  |
postgres_1  | 2020-05-29 17:02:22.804 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgres_1  | 2020-05-29 17:02:22.804 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1  | 2020-05-29 17:02:22.811 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2020-05-29 17:02:22.829 UTC [64] LOG:  database system was shut down at 2020-05-29 17:02:22 UTC
postgres_1  | 2020-05-29 17:02:22.835 UTC [1] LOG:  database system is ready to accept connections
j
OH I think I may know what it is. There’s this https://github.com/PrefectHQ/prefect/blob/90ef46d1200d7ae3ee6811d78671293fa57d18ac/src/prefect/cli/server.py#L278-L283 which I bet could be causing the issue. Could you test something for me and try a run of 
prefect server start
off of an install from this branch:
Copy code
pip install <https://github.com/PrefectHQ/prefect.git@rootles_docker_linux_fix>
And then confirm it’s running off of this brach by running 
prefect version
to see if the output is something like 
0.11.4+7.g846eb1f4f
I’m guessing that it’s trying to add an extra host of something like 
host.docker.internal:""
 since 
get_docker_ip
may be returning nothing
p
Hi Josh. It worked! For future visitors, I had to add 
git+
before the url.
Copy code
pip install git+<https://github.com/PrefectHQ/prefect.git@rootles_docker_linux_fix>
Thanks for your help!
j
Awesome! I’ll get this fixed up and PR’d
p
Hi Josh. Is there not a 
prefect server stop
command?
j
There is not yet, will be soon. At this moment hitting Ctrl + C will stop all containers in that process. Other options would be to clone the repo and run 
docker-compose down
Someone else just asked about this 😄 https://prefect-community.slack.com/archives/CL09KU1K7/p1590781806173900
p
Yeah, a stop command would be great. Also, it would be nice to be able to have the docker compose file more accessible for tweaking.
👍 1
j
Awesome! Maybe there’s room for a 
show
or 
export
command so the yaml file could be output to stdout and worked with
p
And maybe 
import
or 
-f
so you can start a custom docker-compose with 
prefect server start --input <my file>
r
@josh so I need to follow these steps or wait for new release? How does it looks?
j
The release is out and this change it in! 
0.11.5
give it a test!
r
@josh Looks like working... But another error occcured:
j
Hard to diagnose here, would you mind opening an issue on the repo?
👍 1
r
Done!
39 Views
Open in Slack
PreviousNext
Powered by