Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

What tends to cause this error for a task in the middle of a flow, when upstream tasks have run fine? `: No heartbeat detected from the remote task;`

I typed up a FAQ answer <https://docs.prefect.io/orchestration/faq/faq.html#why-does-prefect-mark-flow-runs-with-no-heartbeat-detected-from-the-remote-task-marking-the-run-as-failed|here> for that

<@U01QEJ9PP53> Could you elaborate on this part of your answer? I tried configuring to use threads and still got the same error so now I'm onto the last solution: `In the event that the flow continues to fail after switching to using threads, some users have had success in making the failing task a subflow, and then turning off heartbeats for the subflow.`

If you get the same error, I think it’s highly likely you are running out of memory. Also, how did you set the threaded heartbeats? For this one just one a new Flow with 1 task and then heartbeats off for that, and then call it with the `create_flow_run` task

I added this to my ECS run config at the bottom

```env={"PREFECT__CLOUD__HEARTBEAT_MODE": "thread"}```

Ah yeah that should work. I would suspect you have some memory issue as well and you can try bumping up the resources

Is there a reason I don't get the error when running it from my anaconda terminal but it does happen in production? This also happened when I tested out Prefect's case/when statement structure.

I wouldn't know honestly, I'm just a user. The engineers set it up.

You can show me your ECS task definition or maybe you can get some insight running this with debug level logs?

I'll try the debug logs. What's the best way to do that?

`PREFECT___LOGGING____LEVE_L:"DEBUG"` in the run config as part of the env. you can also show me the RunConfig and task definition if you want and I think we can increase the resources

Task: `
```@task(name = "Transform", log_stdout=True, max_retries=3, retry_delay=timedelta(seconds=10))```


```flow.run_config = ECSRun(
    labels=["ourlabel"],
    image="ourimage",
    env={"PREFECT__CLOUD__HEARTBEAT_MODE": "thread"})```

Sorry task is overloaded lol. I mean ECS Task definition. I think the default is 2GB RAM

I don't think I ever specify the ECS task at any point

Ok let’s try bumping up that memory spec

Can you try filling cpu and memory? There is info in the <https://docs.prefect.io/api/latest/run_configs.html#ecsrun|docstring> to what are acceptable values

How do I see the current memory allocated?

Screen Shot 2022-03-08 at 10.54.46 AM.png

Screen Shot 2022-03-08 at 10.54.57 AM.png

There is a task definition and you can click into it and then you can scroll down

<https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/ecs/task_definition.yaml|This> is the default task definition so those are the default values your flow gets

so I'd be adjusting the memory upward in the ECS run definition at the bottom of my flow?

Yes and that will override the default Prefect gives

Any general rule on how much to increase, if the default is 2084?

No general rule because I know you work with a bunch of dataframes so it depends on the data size