Does anyone know if it’s possible to find out the ECS task ID of a Prefect task from Prefect or the task itself? The context is I am using the ECSAgent to run a task which dumps its logs into a cloudwatch group which is called

prefect/task/{ECS-task-id}

. My aim is to display those logs in our webapp to show admin users the status of the flows they’ve started Ideally I’d be able to use the ID of a flow run and use that to query the ECS task it ran on and then use that to query the logs of the flow

My aim is to display those logs in our webapp to show admin users the status of the flows they’ve started

I'm so glad you mentioned the actual purpose of this, because this way I can show you a much better way of solving this without having to rely on CloudWatch API. Since Prefect already stores all your flow run logs in the backend, you could retrieve those easily in your web API code as follows:

from prefect.backend.flow_run import FlowRunView
flow_run = FlowRunView.from_flow_run_id(
    "9549c55f-1afe-45b6-afb3-1b1ed32e7c3f"
)
logs = flow_run.get_logs()

This will give you logs in this format:

FlowRunLog(timestamp=DateTime(2022, 3, 15, 10, 3, 15, 776868, tzinfo=Timezone('+00:00')), level=20, message='some log msg')

You can see more in this page. I agree this is a lot easier to do on the Prefect side than Cloudwatch

My understanding was that Prefect doesn’t retain logs all that long (7 days or something from memory)? Has that changed? Ideally I want to be able to see logs for my tasks forever

Yeah no we still don’t. No change. You’d need to send your logs to DataDog or a service like that

I’ve just checked a task from October last year and it still has its logs. Is this because I am saving my logs to cloud watch? i.e. is Prefect fetching my logs from there and deleting them internally, or am I just lucky?

Are you on Server or Cloud?

Cloud