Hi all! I've got a weird issue when starting the p...
# prefect-server
Hi all! I've got a weird issue when starting the prefect server. If anyone have ever faced this issue before, please help me.
Copy code
WARNING: The PREFECT_SERVER_DB_CMD variable is not set. Defaulting to a blank string.
WARNING: The DB_CONNECTION_URL variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_DB variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_USER variable is not set. Defaulting to a blank string.
Creating network "prefect-server" with the default driver
Creating tmp_postgres_1 ... done
Creating tmp_hasura_1   ... done
Creating tmp_graphql_1  ... done
Creating tmp_towel_1    ... done
Creating tmp_apollo_1   ... done
Creating tmp_ui_1       ... done
Attaching to tmp_postgres_1, tmp_hasura_1, tmp_graphql_1, tmp_towel_1, tmp_apollo_1, tmp_ui_1
graphql_1   | bash: -c: line 0: syntax error near unexpected token `&&'
graphql_1   | bash: -c: line 0: ` && python src/prefect_server/services/graphql/server.py'
hasura_1    | {"type":"pg-client","timestamp":"2020-10-26T11:45:30.504+0000","level":"warn","detail":{"message":"postgres connection failed, retrying(0)."}}
hasura_1    | {"type":"pg-client","timestamp":"2020-10-26T11:45:30.504+0000","level":"warn","detail":{"message":"postgres connection failed, retrying(1)."}}
hasura_1    | {"type":"startup","timestamp":"2020-10-26T11:45:30.504+0000","level":"error","detail":{"kind":"db_migrate","info":{"internal":"could not connect to server: No such file or directory\n\tIs the server running locally and accepting\n\tconnections on Unix domain socket \"/var/run/postgresql/.s.PGSQL.5432\"?\n","path":"$","error":"connection error","code":"postgres-error"}}}
ui_1        | πŸ‘ΎπŸ‘ΎπŸ‘Ύ UI running at localhost:8080 πŸ‘ΎπŸ‘ΎπŸ‘Ύ
postgres_1  | Error: Database is uninitialized and superuser password is not specified.
postgres_1  |        You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1  |        superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1  | 
postgres_1  |        You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1  |        connections without a password. This is *not* recommended.
postgres_1  | 
postgres_1  |        See PostgreSQL documentation about "trust":
postgres_1  |        <https://www.postgresql.org/docs/current/auth-trust.html>
tmp_hasura_1 exited with code 1
tmp_graphql_1 exited with code 1
tmp_postgres_1 exited with code 1
Your postgres container is failing to initialize because the POSTGRES_PASSWORD variable is not set correctly.
Copy code
(venv) shark@shark-H310M-S2-2-0:~/Documents/prefect$ prefect config

{'debug': False, 'home_dir': '/home/shark/.prefect', 'backend': 'cloud', 'server': {'host': '<http://localhost>', 'port': 4200, 'host_port': 4200, 'endpoint': '<http://localhost:4200>', 'database': {'host': 'localhost', 'port': 5432, 'host_port': 5432, 'name': 'prefect_server', 'username': 'postgres', 'password': 'postgres', 'connection_url': '<postgresql://postgres:postgres@localhost:5432/prefect_server>', 'volume_path': '/home/shark/.prefect/pg_data/'}, 'graphql': {'host': '', 'port': 4201, 'host_port': 4201, 'debug': False, 'path': '/graphql/'}, 'hasura': {'host': 'localhost', 'port': 3000, 'host_port': 3000, 'admin_secret': '', 'claims_namespace': 'hasura-claims', 'graphql_url': '<http://localhost:3000/v1alpha1/graphql>', 'ws_url': '<ws://localhost:3000/v1alpha1/graphql>', 'execute_retry_seconds': 10}, 'ui': {'host': '<http://localhost>', 'port': 8080, 'host_port': 8080, 'endpoint': '<http://localhost:8080>', 'apollo_url': '<http://localhost:4200/graphql>'}, 'telemetry': {'enabled': True}}, 'cloud': {'api': '<https://api.prefect.io>', 'endpoint': '<https://api.prefect.io>', 'graphql': '<https://api.prefect.io/graphql>', 'use_local_secrets': True, 'heartbeat_interval': 30.0, 'check_cancellation_interval': 15.0, 'diagnostics': False, 'logging_heartbeat': 5, 'queue_interval': 30.0, 'agent': {'name': 'agent', 'labels': [], 'level': 'INFO', 'auth_token': '', 'agent_address': '', 'resource_manager': {'loop_interval': 60}}}, 'logging': {'level': 'INFO', 'format': '[%(asctime)s] %(levelname)s - %(name)s | %(message)s', 'log_attributes': [], 'datefmt': '%Y-%m-%d %H:%M:%S', 'log_to_cloud': False, 'extra_loggers': []}, 'flows': {'eager_edge_validation': False, 'run_on_schedule': True, 'checkpointing': False, 'defaults': {'storage': {'add_default_labels': True, 'default_class': 'prefect.environments.storage.Local'}}}, 'tasks': {'defaults': {'max_retries': 0, 'retry_delay': None, 'timeout': None}}, 'engine': {'executor': {'default_class': 'prefect.engine.executors.LocalExecutor', 'dask': {'address': '', 'cluster_class': 'distributed.deploy.local.LocalCluster'}}, 'flow_runner': {'default_class': 'prefect.engine.flow_runner.FlowRunner'}, 'task_runner': {'default_class': 'prefect.engine.task_runner.TaskRunner'}}}
This is the config of the prefect. I'm not sure what is wrong in there.
Actually, I did nothing, I just installed Preffect with Conda and set backend as server(
prefect backend server
) and start the prefect server running this command:
prefect server start
I was running into this error very frequently a couple of weeks ago, but it was because I was trying to build my own set of container definitions. Using
prefect server start
should mean that you're using the default
, which lives here https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/docker-compose.yml
My guess is that you have setup a local
file that doesn't have a postgres password set
I tried to make the toml file to fix the issue as well. This is the current config.toml file.
Copy code
        username = "postgres"
        password = "postgres"
        volume_path = "~/.prefect/pg_data/"
Is there something wrong, @Billy McMonagle?
I'm not sure. The problem is specific to the postgres container. you might want to also set a value for $POSTGRES_DB. In the .toml file, you can add the key
name = "your_db_name
(I'm only sharing my experience since I had a similar issue, I am not affiliated w Prefect πŸ™‚ )
Yeah, thank you so much, @Billy McMonagle However, I don't think the problem is not associated with missing POSTGRES_DB in the config.toml file, you know.
I think there must be another reason, you know.
why don't you do this... my guess is that maybe one of the other containers is making an assumption about your user/password values...
I used these values with success, for local testing:
Copy code
parameter :postgres_user, type: :String, default: "prefect"
parameter :postgres_password, type: :String, default: "test-password"
parameter :postgres_db, type: :String, default: "prefect_server"
If I don't set username and password for the postgres in the config.toml file, do you think it works then?
I don't use my own docker-compose.yml file. I just use default compose file, you know
I don't think so because I think you need to set a default value somewhere
you could try making a copy of the docker-compose file in your project directly and running it directly, without going through the
CLI command.
well... I don't think it's the right way to fix the issue.
up to you. I don't plan to use docker-compose in production at all. Β―\_(ツ)_/Β―
πŸ‘ 1
prefect was already providing the default values for the postgres db conntion, actually.
does that mean it's working for you?
as you can see the above prefect config in JSON, values of the Postgres DB connection fields seem to be correct ( they're not empty ). but the error is raised when the postgres container is created through docker.
I already tried to add the POSTGRES_USER in the config.toml file like the bellow and run prefect server, but was not working.
Copy code
        name = "prefect_server"
        username = "postgres"
        password = "postgres"
        volume_path = "~/.prefect/pg_data/"
you might want to do a
docker system prune
to cleanup your local system.
bc you are using a local volume, i'm wondering if you just need to wipe the data and start a fresh one
I also found it useful to run just the pg container directly, it helps to minimize the number of moving pieces. eg
docker run --rm   --name pg-docker -e POSTGRES_PASSWORD=password -p 5432:5432 postgres
Thanks, Billy. let me try to clean everything(
docker system prune -a
) and run it again. although I already did it before adding the
property to the config.toml file.
good luck! this can be frustrating, I only know docker well enough to get myself in trouble sometimes.
yeah, I understand.
Actually, this strange docker issue gets me stick in the same place for a long time.
yeah, running the postgres image directly works now!
πŸŽ‰ 1
hmm... but, how can we always run the prefect server like that?
run it from .
local/bin/prefect server start
not directly in your home, it will then use the toml. Search and read my comments here from the last few days, encountered the same problem... docker-compose will probably need a
Thanks for your message, Jasper. Let me play with that and let you know.
However, I can not find
directory within
Copy code
(venv) shark@shark-H310M-S2-2-0:~/Documents/prefect$ ~/.local/
lib/   share/