m

    Max

    1 year ago
    Hi everyone! I'm trying to setup Prefect with Kubernetes on my local machines. I managed to start all the required pods (
    postgres, towel, hasura, graphql, apollo, ui
    ), but from the logs it appears that something is wrong with either hasura or graphsql (the logs are in the thread). What could be the issue? How does one even debug this kind of errors?
    $ kubectl logs towel-7cbcb75c49-w242j
    {"severity": "ERROR", "name": "prefect-server.ZombieKiller", "message": "Unexpected error: ValueError([{'extensions': {'path': '$.selectionSet.task_run', 'code': 'validation-failed'}, 'message': 'field \"task_run\" not found in type: \\'query_root\\''}])"}
    
    $ kubectl logs hasura-58bf695f95-qnwpp
    {"type":"http-log","timestamp":"2020-10-30T18:14:36.200+0000","level":"error","detail":{"operation":{"user_vars":{"x-hasura-role":"admin"},"error":{"path":"$.selectionSet.task_run","error":"field \"task_run\" not found in type: 'query_root'","code":"validation-failed"},"request_id":"5126b8cc-d17f-4833-b9b7-5ecdcf56c9de","response_size":122,"query":{"variables":{},"query":"query {\n    task_run(where: { state: { _eq: \"Running\" }, heartbeat: { _lte: \"2020-10-30T18:04:36.189977+00:00\" }, task: { flow: { flow_group: { _not: { settings: { _contains: { heartbeat_enabled: false } } } } } } }, order_by: { updated: desc }, limit: 5000) {\n        id\n        flow_run_id\n        tenant_id\n        flow_run {\n            state\n        }\n        task {\n            retry_delay\n            max_retries\n        }\n        retry_count: states_aggregate(where: { state: { _eq: \"Retrying\" } }) {\n            aggregate {\n                count\n            }\n        }\n    }\n}"}},"http_info":{"status":400,"http_version":"HTTP/1.1","url":"/v1alpha1/graphql","ip":"192.168.188.166","method":"POST","content_encoding":null}}}
    Dylan

    Dylan

    1 year ago
    Hi @Max, Welcome to the Community!
    It looks like the particular error you’re experiencing may be happening because: a. Towel spun up before Apollo + graphql were ready and therefore is looking for something that doesn’t exist in the graphql schema b. migrations haven’t run in hasura
    m

    Max

    1 year ago
    thanks for such a quick reply! I'll take a look 🙂
    Dylan

    Dylan

    1 year ago
    actually that is for the older major version of Prefect Server
    Wrong doc, sorry
    Let me find the correct one
    This is the one I’m thinking of 👍
    m

    Max

    1 year ago
    I finally managed to make it work based on this example, thank you so much!
    Sagun Garg

    Sagun Garg

    1 year ago
    @Dylan Sorry for jumping in this thread in between, just wanted to check if I can manually trigger Hasura migrations as I used AWS ECS Fargate setup using docker-compose.yml and getting errors in towel container as shared below. @Max I did a setup in AWS Faragte ECS and I am getting similar container logs for Towel recently, how did you fix the same https://prefect-community.slack.com/archives/C014Z8DPDSR/p1612113375123600