Jesper van Dijke
10/20/2020, 8:38 PMnew user: prefect , added to sudo and docker, nothing special regarding environment.
docker was already installed, I did add docker-compose
stopped my running postgres service, so it doesnt have any port conflicts, however would love to use this one.
---> This creates a .local in my prefect home, navigate to /bin and /lib , took a closer look, runpip install prefect
---> creates a .prefect with a .toml file with setting : backend = "server" , so far so good I guess. Now comes the next step in the documentation, runprefect backend server
---> Starts popping warning regarding postgresprefect server start
WARNING: The PREFECT_SERVER_DB_CMD variable is not set. Defaulting to a blank string.
WARNING: The DB_CONNECTION_URL variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_DB variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_USER variable is not set. Defaulting to a blank string.
Pulling postgres ... done
Pulling hasura ... done
Pulling graphql ... done
Pulling apollo ... done
Pulling towel ... done
Pulling ui ... done
WARNING: The PREFECT_SERVER_DB_CMD variable is not set. Defaulting to a blank string.
WARNING: The DB_CONNECTION_URL variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_DB variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_USER variable is not set. Defaulting to a blank string.
Creating network "prefect-server" with the default driver
Creating tmp_postgres_1 ... done
Creating tmp_hasura_1 ... done
Creating tmp_graphql_1 ... done
Creating tmp_towel_1 ... done
Creating tmp_apollo_1 ... done
Creating tmp_ui_1 ... done
Attaching to tmp_postgres_1, tmp_hasura_1, tmp_graphql_1, tmp_towel_1, tmp_apollo_1, tmp_ui_1
graphql_1 | bash: -c: line 0: syntax error near unexpected token `&&'
graphql_1 | bash: -c: line 0: ` && python src/prefect_server/services/graphql/server.py'
hasura_1 | {"type":"pg-client","timestamp":"2020-10-20T20:15:35.282+0000","level":"warn","detail":{"message":"postgres connection failed, retrying(0)."}}
hasura_1 | {"type":"pg-client","timestamp":"2020-10-20T20:15:35.282+0000","level":"warn","detail":{"message":"postgres connection failed, retrying(1)."}}
hasura_1 | {"type":"startup","timestamp":"2020-10-20T20:15:35.282+0000","level":"error","detail":{"kind":"db_migrate","info":{"internal":"could not connect to server: No such file or directory\n\tIs the server running locally and accepting\n\tconnections on Unix domain socket \"/var/run/postgresql/.s.PGSQL.5432\"?\n","path":"$","error":"connection error","code":"postgres-error"}}}
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | <https://www.postgresql.org/docs/current/auth-trust.html>
So where to start from here, adjusting docker compose file or other way to configure the local env?emre
10/20/2020, 9:03 PM~/.prefect/config.toml
, its path and name can be customized by the env var PREFECT__USER_CONFIG_PATH
.
Check the config.toml
included in the prefect repo, as a default to start with, for most if not all options you can tinker with, :
https://github.com/PrefectHQ/prefect/blob/master/src/prefect/config.toml
Also checking the cli for the server
subcommand and the docker-compose.yml
helped me a lot when setting up a server:
https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/server.py
https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/docker-compose.ymlJesper van Dijke
10/20/2020, 9:20 PMconfig.toml
are these settings username and password used while running the docker-compose? Within the yml service: image: "postgres:11"
?
I get confused.emre
10/20/2020, 9:39 PMPOSTGRES_PASSWORD
for the postgres container. Therefore, when it is time for prefect server
to start docker-compose, these environment variables must be present.
prefect server
subcommand temporarily sets env vars such as POSTGRES_PASSWORD
with values read from the config file: config.toml
.
If a config file cannot be found, or a requested config parameter isn’t in the file, I believe it takes a None
value. This means prefect sets POSTGRES_PASSWORD
to an empty value. This I believe is why your postgres container had an empty password.Jesper van Dijke
10/20/2020, 10:20 PMcmd = ["docker-compose", "up"]
proc = subprocess.Popen(cmd, cwd=compose_dir_path, env=env)
it should have the env vars,docker-compose
... pretty annoying.
https://prefect-community.slack.com/archives/CL09KU1K7/p1603235984359800