https://prefect.io
Join Slack
Been receiving these error messages. Weird! Can anyone explain? ```Submission failed. kubernetes.cli...
# ask-community
d
Been receiving these error messages. Weird! Can anyone explain?
Copy code
Submission failed. kubernetes.client.exceptions.ApiException: (500) Reason: Internal Server Error HTTP response headers: HTTPHeaderDict({'Audit-Id': '6d00e224-73e9-4d44-bfce-d0589101f116', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Warning': '299 - "unknown field \\"spec.template.spec.parallelism\\"", 299 - "unknown field \\"spec.template.spec.completions\\""', 'X-Kubernetes-Pf-Flowschema-Uid': 'e8e112c2-396a-440d-a000-51271ccde7dd', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'a49f8ad5-8431-4216-90ae-cd35c64b0736', 'Date': 'Wed, 10 May 2023 18:30:59 GMT', 'Content-Length': '161'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"rpc error: code = Unknown desc = exec (try: 500): database is locked","code":500}
n
hi @Deceivious - are you running against Prefect Cloud?
d
Nope
well atleast this issue isnt in cloud
we use both version
z
Hm that looks like a Kubernetes internal error?
d
Thanks ill scope it out tomorrow and if I discover something ill let you know if its related to prefect
Ummm @Nate, @Zanie Unsure if this is an issue. But the status is shown as "Failed" and not "Crashed". At this point when the pod itself failed to spawn, , the 
on_failure
hook will not get triggered because the pod hasnt started up. Should the status be something else other than failure?
Is there any way of getting notified about such issues?
n
we were just discussing this, we think this should be a 
Crashed
state - this situation is a good example of why opening issue now
d
Thanks. Please link the issue here as well so i can follow it.
But the no notification issue will still exist i guess because there is no infrastucture to run 
on_failure
and 
on_crash
hook. At which point, the only way would be to create an automation ?
z
Yeah automations are the only way to capture that kind of failure
n
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by