Hi everyone! I'm trying to setup Prefect with Kub...
# prefect-server
Hi everyone! I'm trying to setup Prefect with Kubernetes on my local machines. I managed to start all the required pods (
postgres, towel, hasura, graphql, apollo, ui
), but from the logs it appears that something is wrong with either hasura or graphsql (the logs are in the thread). What could be the issue? How does one even debug this kind of errors?
Copy code
$ kubectl logs towel-7cbcb75c49-w242j
{"severity": "ERROR", "name": "prefect-server.ZombieKiller", "message": "Unexpected error: ValueError([{'extensions': {'path': '$.selectionSet.task_run', 'code': 'validation-failed'}, 'message': 'field \"task_run\" not found in type: \\'query_root\\''}])"}

$ kubectl logs hasura-58bf695f95-qnwpp
{"type":"http-log","timestamp":"2020-10-30T18:14:36.200+0000","level":"error","detail":{"operation":{"user_vars":{"x-hasura-role":"admin"},"error":{"path":"$.selectionSet.task_run","error":"field \"task_run\" not found in type: 'query_root'","code":"validation-failed"},"request_id":"5126b8cc-d17f-4833-b9b7-5ecdcf56c9de","response_size":122,"query":{"variables":{},"query":"query {\n    task_run(where: { state: { _eq: \"Running\" }, heartbeat: { _lte: \"2020-10-30T18:04:36.189977+00:00\" }, task: { flow: { flow_group: { _not: { settings: { _contains: { heartbeat_enabled: false } } } } } } }, order_by: { updated: desc }, limit: 5000) {\n        id\n        flow_run_id\n        tenant_id\n        flow_run {\n            state\n        }\n        task {\n            retry_delay\n            max_retries\n        }\n        retry_count: states_aggregate(where: { state: { _eq: \"Retrying\" } }) {\n            aggregate {\n                count\n            }\n        }\n    }\n}"}},"http_info":{"status":400,"http_version":"HTTP/1.1","url":"/v1alpha1/graphql","ip":"","method":"POST","content_encoding":null}}}
Hi @Max, Welcome to the Community!
It looks like the particular error you’re experiencing may be happening because: a. Towel spun up before Apollo + graphql were ready and therefore is looking for something that doesn’t exist in the graphql schema b. migrations haven’t run in hasura
thanks for such a quick reply! I'll take a look 🙂
actually that is for the older major version of Prefect Server
Wrong doc, sorry
Let me find the correct one
This is the one I’m thinking of 👍
I finally managed to make it work based on this example, thank you so much!
@Dylan Sorry for jumping in this thread in between, just wanted to check if I can manually trigger Hasura migrations as I used AWS ECS Fargate setup using docker-compose.yml and getting errors in towel container as shared below. @Max I did a setup in AWS Faragte ECS and I am getting similar container logs for Towel recently, how did you fix the same https://prefect-community.slack.com/archives/C014Z8DPDSR/p1612113375123600