You shouldn't need extra AWS-side setup like CloudWatch Container Insights for those graphs. For Prefect 3.x, the new Infrastructure panel compute graphs are collected by the Prefect runtime inside the flow run container/process itself, then sent to Prefect Cloud via telemetry. So for ECS, the main requirements are: * You’re running in Prefect Cloud (not self-hosted server) * The flow run container has a valid

PREFECT_API_URL

and

PREFECT_API_KEY

* Resource metrics telemetry is enabled * The image includes the OpenTelemetry system metrics dependency From what you described,

prefecthq/prefect-aws:latest

should already have the needed dependency, so in most cases there’s no separate AWS config required. What to check 1. Confirm metrics telemetry is enabled These settings control it:

PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS=true
   PREFECT_TELEMETRY_RESOURCE_METRICS_INTERVAL_SECONDS=10

The first one is the important one. It defaults to enabled, but if it’s been turned off anywhere, the graphs won’t appear. 2. Confirm the flow run container gets Prefect Cloud credentials The ECS task that actually runs the flow needs:

PREFECT_API_URL=...
   PREFECT_API_KEY=...

3. Check task logs with debug logging enabled If metric collection fails to initialize, Prefect may just skip the graphs. Try setting:

PREFECT_LOGGING_LEVEL=DEBUG

Then look for logs mentioning telemetry / metrics initialization. 4. Make sure you’re looking at a new flow run These metrics are collected during execution, so older runs from before the rollout/version change won’t show them. 5. Possible rollout / UI-side caveat Since this feature landed recently, it may also be partially rollout-gated or only visible for certain run types/pages. If everything above looks right and new ECS runs still show no graphs, that points more to a Prefect Cloud/UI-side issue than AWS configuration. Short answer to your AWS question: No, you generally do not need to enable CloudWatch Container Insights or other ECS metrics settings for Prefect’s Infrastructure panel graphs. Relevant docs/code context: * Infrastructure debugging docs: https://docs.prefect.io/ * Prefect releases: Prefect releases If you want, I can help you narrow this down by checking: * whether your worker env vs flow task env has the telemetry vars * whether your ECS job template is overriding env/image settings * what logs to look for in a specific run

That sounds like a different feature. “Metrics powered runs page” refers to the newer Runs page analytics/experience, while the Infrastructure panel compute graphs are tied to resource metrics collection for individual flow runs. So, in short: * Metrics powered runs page toggle = affects the runs page UI/analytics experience * Infrastructure panel CPU/memory graphs = depends on run resource telemetry being collected and rendered for that run They’re related in the broad sense that both use metrics, but they’re not the same switch as far as the implementation goes. Given your symptom — Submitting state appears, but no Infrastructure graphs — I’d be more suspicious of one of these: * telemetry disabled in the run environment * the flow run container not exporting metrics successfully * a Cloud/UI rollout gap or bug for ECS-backed runs If you want, I can help you verify the exact env vars your ECS flow-run task is getting and what log lines would confirm metrics export is active.

Thanks — that extra detail is very helpful. The most important point is: resource metrics are not sent as Prefect events So the fact that you only see state changes and

prefect.flow-run.heartbeat

events with empty payloads is not itself evidence that metrics are missing. The Infrastructure panel graphs do not come from the events stream. They’re sent separately via OpenTelemetry/OTLP to Prefect Cloud. How it’s supposed to work For Prefect 3.x, resource monitoring works roughly like this: 1. The flow run starts through the Prefect engine 2. Prefect wraps execution in an internal metrics context 3. It uses OpenTelemetry system metrics instrumentation to sample process CPU/memory on an interval 4. Those metrics are exported directly to Prefect Cloud at a telemetry endpoint 5. The UI renders the Infrastructure panel charts from that telemetry data So in a healthy setup, you would not expect to see resource metric events in the event feed. What this means for your case Given your findings, the key clues are: * you’re on recent Prefect/`prefect-aws` * ECS observer works * the flow task command is

prefect flow-run execute

* telemetry env vars are set * heartbeats are normal * but there are no obvious telemetry/resource log lines That suggests one of these is more likely: 1. The flow run process is not entering the code path that initializes resource metrics 2. The telemetry dependency is missing in the actual flow run image 3. Metrics export is failing silently / being skipped 4. There is a Cloud-side/UI limitation or bug affecting ECS/Fargate runs One especially important detail: custom task definitions You mentioned custom task definition. That matters because resource metrics are only collected when the flow run is executed through the normal Prefect engine path. If your custom task definition changes entrypoint/command behavior in a way that bypasses the engine bootstrap, metrics can disappear even if the run itself works. You said the command is:

prefect flow-run execute

That sounds right, but I’d still verify the actual container image and runtime environment of the flow-run task, not just the worker/service. In particular: * Is the flow-run container definitely using an image with the OpenTelemetry system metrics package installed? * Does the custom task definition set its own entrypoint? * Are you passing a full task definition ARN/template that might override what Prefect would normally inject? What “healthy” should look like There isn’t much visible signaling today. From the implementation, metrics export is fairly transparent and can fail quietly. So a healthy run may show: * no special metric events * few or no obvious telemetry log lines * CPU/memory graphs appearing in the Infrastructure panel after some delay That means there’s unfortunately no great “demo trace” in logs to compare against. What it is not dependent on As far as the implementation goes, this does not appear to be limited to a specific compute type like: * ECS EC2 vs Fargate * Kubernetes * Docker It should work for ECS/Fargate too, provided the flow run is launched through the supported Prefect engine path and the image has the required telemetry bits. Most likely next checks I’d focus on these: 1. Verify the actual flow-run image contents In the flow-run task image, confirm the OpenTelemetry dependency is present. If it’s missing, Prefect can skip resource metrics collection. 2. Compare with a non-custom deployment path If possible, run one test deployment using the most standard ECS worker/task setup you can, without the custom task definition overrides, and see whether graphs appear. If they do, that strongly implicates the custom task definition/runtime path. 3. Check whether the flow-run task can reach Prefect Cloud telemetry endpoint Metrics are sent to the Prefect Cloud API telemetry endpoint, not the websocket/events channel.

4. Try with the official Prefect image for the flow-run task Even for one short test run, using

prefecthq/prefect-aws:latest

directly for the flow run can help isolate whether this is image-specific. Bottom line Based on what you shared, I do not think the absence of resource metric events is the issue, because metrics are not event-based. The more relevant concern is that the flow-run process is not successfully initializing or exporting OTLP resource metrics, and a custom ECS task definition is one plausible reason. I also checked for existing public issues/discussions and didn’t find a clear match, which makes this look like either: * an undocumented caveat with ECS/Fargate/custom task definitions, or * a regression/bug If you want, I can help you with a very targeted validation checklist for the ECS task definition/image — basically the 5 things to inspect in the task JSON/container config that would most likely explain why metrics never appear.

Yes — that’s an expected limitation. Short answer: Prefect’s Infrastructure panel is collecting metrics for the Prefect flow-run process itself, not the full process tree. So child/grandchild processes launched via

subprocess.run()

, shell scripts, or a JVM are not included in the CPU/memory numbers shown there. That would explain exactly what you’re seeing: * Prefect chart: ~0.3% CPU * CloudWatch ECS task: near task CPU limit Those are measuring different scopes. What Prefect is measuring Prefect 3.x uses OpenTelemetry system/process metrics instrumentation for the process running the flow. The collected metrics are the standard process-level ones, including: *

process.cpu.utilization

*

process.memory.usage

*

process.memory.virtual

This is not container-level aggregation, and Prefect does not add custom “sum all descendants” logic on top. So if your flow process mostly just: * calls

subprocess.run(...)

* waits for a shell script * which launches a JVM * and the JVM does the real work …then Prefect’s own process can remain almost idle while the container as a whole is very busy. What CloudWatch is measuring ECS/CloudWatch task metrics are generally much closer to container/task-level resource usage, so they will reflect the JVM and any other child processes in the same container/task. That’s why CloudWatch is the better source for “how much CPU did this ECS task actually consume?”, while Prefect’s panel is better interpreted as “how busy was the Prefect runner process itself?” Is child/grandchild inclusion supported? From the implementation/semantics: no, not by default. So: * child processes: not included * grandchild processes: not included * JVM spawned by a shell script: not included Is this a known limitation? Yes — I’d characterize it as an inherent limitation of the current process-level metric collection approach rather than something specific to ECS. Practical takeaway If your workload pushes most compute into subprocesses, then: * Prefect Infrastructure graphs will underreport actual task/container usage * CloudWatch ECS metrics are the right source for true compute consumption * This discrepancy is expected for subprocess-heavy workloads Rule of thumb * Use Prefect graphs for Python process behavior * Use CloudWatch / ECS metrics for full container/task behavior If you want, I can also help you think through a couple of ways to make this more observable, for example: * emitting your own custom metrics around the subprocess/JVM * logging child PID/resource info * splitting the heavy binary/JVM into its own observable unit so ECS metrics line up more cleanly with Prefect runs

Short answer: No — switching to

prefect-shell

will not solve this.

prefect-shell

still runs the shell command as a child process of the flow-run Python process. Prefect’s Infrastructure panel metrics remain scoped to the main flow-run process, so the shell/JVM work still won’t be included. So these two cases are effectively the same from the metrics perspective: *

subprocess.run(...)

inside your flow *

prefect-shell

task/block launching bash In both cases, the heavy work happens in descendant processes, and Prefect’s built-in resource graphs won’t aggregate that usage. What would be required for

prefect-shell

to help? There really aren’t “prerequisites” that make built-in Prefect metrics suddenly include the JVM. The limitation is architectural: * Prefect collects process-level metrics * not process tree metrics * not container/task-level metrics So even with

prefect-shell

, the JVM process would still be outside the default scope. Is it simply not possible to collect metrics for JVM processes? With Prefect’s built-in Infrastructure panel, effectively: not today, at least not automatically for subprocess/grandchild JVMs. But it is possible to collect JVM metrics more broadly — just not via the current built-in Prefect process metrics alone. Your realistic options are: 1. Use ECS/CloudWatch metrics Best if you want total container/task CPU and memory. This already matches your workload better. 2. Instrument the JVM separately For example with JVM/OpenTelemetry tooling, JMX exporters, or another monitoring path. This gives true JVM-level metrics, but outside Prefect’s default Infrastructure chart behavior. 3. Add custom metrics/logging from the flow You could sample the child PID tree yourself and emit logs or external metrics correlated with the flow run. 4. Restructure so the heavy compute happens in-process Only if practical. If the Python flow process itself does the heavy work, Prefect’s charts will reflect it. But for a JVM launched by shell, that usually isn’t realistic. Best mental model * Prefect Infrastructure panel = “how busy was the Prefect runner Python process?” * CloudWatch ECS metrics = “how busy was the ECS task/container?” * JVM metrics tooling = “how busy was the Java process?” For your use case, CloudWatch or JVM-native instrumentation is the right source of truth. If you want, I can suggest a lightweight pattern to correlate ECS/CloudWatch task metrics back to a Prefect flow run ID, which is usually the most practical workaround for subprocess-heavy jobs.