Dear prefect people, we get plenty of

No heartbeat detected from the remote task; marking the run as failed.

errors. 💔 (e.g. https://cloud.prefect.io/accure/task-run/8e44bd11-e7c3-4982-bcf2-7711bc1ca4a9?logId=83475348-2531-4efb-9663-33748515908e) This is particularly unexpected, as most of those tasks seem to have already almost finished... 🤔 How do you approach these errors and debug them? 🙂

I suppose for starters you can disable heartbeat checks and see if your flow is still facing any issues - you can toggle off heartbeat from the UI or using graphql ...

Hi Robin! Marwan’s suggestion is a good one. Also, sometimes users experience this issue when their tasks are resource-starved and the thread that sends heartbeats has trouble getting resources. So you might try increasing resources to your tasks and see if the issue resolves

OK, merci!

You’ll find it on the Settings page for that Flow

LOL, since when does that one exist? 😄 Never saw it 😮

I think we added it in February 😉

OK, well, better late than never 🙂 But might also be worth to make it more visible?

I’ll definitely pass your feedback along 👍

Haha, thanks for helping out! 🙂

What does your In Progress Flow Runs tile showing on the Dashboard?

Please find attached the screenshots, showing no flows running, two agents querying and 10/13 task concurrency 🤷‍♂️

Hey @Robin Looks like you might have some tasks left in a running state in a cancelled flow run which is causing the issue

OK, where can I find the related commands?

Hi @Robin if you visit the Interactive API page you’ll find the full API documenation

OK, thanks for pointing to the docs. However, I don't understand from it how to e.g. show all running tasks 🤔

You’ll want to construct a query that looks something like this:

query {
  task_run(where: {state: {_eq: "Running"}}) {
    id
    flow_run {
      id
      name
    }
  }
}

Thanks a lot for the additional information!! 🙂

Anytime!

I just also asked for a kill switch feature in general and created this kill switch feature request, as suggested in this discussion.