https://prefect.io logo
Title
v

Vivek Kaushal

04/27/2022, 12:33 PM
Hello folks, I am trying to run Prefect server on an EC2 instance using the instructions in Prefect Docs and am facing the following error while trying to get it up and running. Has anyone faced a similar issue in hosting Prefect server?
Could not upgrade the database!
graphql_1   | Error: HTTPConnectionPool(host='hasura', port=3000): Max retries exceeded with url: /v1/query (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f25746773d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
apollo_1    | Checking GraphQL service at <http://graphql:4201/health> ...
towel_1     | {"severity": "ERROR", "name": "prefect-server.Lazarus", "message": "Unexpected error: ValueError([{'extensions': {'path': '$.selectionSet.flow_run', 'code': 'validation-failed'}, 'message': 'field \"flow_run\" not found in type: \\'query_root\\''}])", "exc_info": "Traceback (most recent call last):\n  File \"/prefect-server/src/prefect_server/services/loop_service.py\", line 60, in run\n    await self.run_once()\n  File \"/prefect-server/src/prefect_server/services/towel/lazarus.py\", line 37, in run_once\n    return await self.reschedule_flow_runs()\n  File \"/prefect-server/src/prefect_server/services/towel/lazarus.py\", line 89, in reschedule_flow_runs\n    limit=500,\n  File \"/prefect-server/src/prefect_server/database/orm.py\", line 501, in get\n    as_box=not apply_schema,\n  File \"/prefect-server/src/prefect_server/database/hasura.py\", line 85, in execute\n    as_box=as_box,\n  File \"/prefect-server/src/prefect_server/utilities/graphql.py\", line 84, in execute\n    raise ValueError(result[\"errors\"])\nValueError: [{'extensions': {'path': '$.selectionSet.flow_run', 'code': 'validation-failed'}, 'message': 'field \"flow_run\" not found in type: \\'query_root\\''}]"}
a

Anna Geller

04/27/2022, 12:43 PM
Yup, just yesterday, someone had the same issue - looks like your Hasura/Prefect Server versions are not matched. Chech: • this page • and this one
v

Vivek Kaushal

04/27/2022, 12:50 PM
Thanks @Anna Geller ! A docker image of Hasura was pulled by the
prefect server start
command though, so a version mismatch should ideally not happen, right?
a

Anna Geller

04/27/2022, 1:08 PM
can you share the output of
prefect diagnostics
? btw are you just getting started with Prefect? perhaps you can start directly with 2.0
v

Vivek Kaushal

04/27/2022, 1:09 PM
Sure Anna, here’s the output:
{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "Linux-5.10.109-104.500.amzn2.x86_64-x86_64-with-glibc2.2.5",
    "prefect_backend": "server",
    "prefect_version": "1.2.0",
    "python_version": "3.7.10"
  }
}
Our usecase is currently critical for production, hence we are planning on sticking to Prefect 1.0, since 2.0 will be in beta for the coming few months. Will be actively following the development though!
a

Anna Geller

04/27/2022, 1:11 PM
thanks. If you check your docker images, what Prefect and Hasura image versions do you see? this error occurs due to a version mismatch so you may remove any old images and try again
if it's critical for production, you may also sign up for Prefect Cloud - there is a free tier of 20,000 task runs per month and you don't have to manage Server (issues like this one) - just mentioning this so that you know this is an option
v

Vivek Kaushal

04/27/2022, 1:16 PM
If you check your docker images, what Prefect and Hasura image versions do you see?
I see
core-1.2.0
as the prefect version and
v2.1.1
as the hasura version. The issue still persists though.
you may also sign up for Prefect Cloud
Thanks for the suggestion Anna, looking into cloud as an option as well!
👍 1
a

Anna Geller

04/27/2022, 1:25 PM
did you try setting this environment variable before starting Server?
export HASURA_GRAPHQL_V1_BOOLEAN_NULL_COLLAPSE=true
k

Kevin Kho

04/27/2022, 1:41 PM
I think the migrations on the database did not run here? You can check this and try to run the migrations