https://prefect.io logo
#prefect-community
Title
# prefect-community
e

Eddie Atkinson

03/30/2022, 11:13 AM
Does anyone know if it’s possible to find out the ECS task ID of a Prefect task from Prefect or the task itself? The context is I am using the ECSAgent to run a task which dumps its logs into a cloudwatch group which is called
prefect/task/{ECS-task-id}
. My aim is to display those logs in our webapp to show admin users the status of the flows they’ve started Ideally I’d be able to use the ID of a flow run and use that to query the ECS task it ran on and then use that to query the logs of the flow
a

Anna Geller

03/30/2022, 11:27 AM
My aim is to display those logs in our webapp to show admin users the status of the flows they’ve started
I'm so glad you mentioned the actual purpose of this, because this way I can show you a much better way of solving this without having to rely on CloudWatch API. Since Prefect already stores all your flow run logs in the backend, you could retrieve those easily in your web API code as follows:
Copy code
from prefect.backend.flow_run import FlowRunView
flow_run = FlowRunView.from_flow_run_id(
    "9549c55f-1afe-45b6-afb3-1b1ed32e7c3f"
)
logs = flow_run.get_logs()
This will give you logs in this format:
Copy code
FlowRunLog(timestamp=DateTime(2022, 3, 15, 10, 3, 15, 776868, tzinfo=Timezone('+00:00')), level=20, message='some log msg')
upvote 1
👀 1
k

Kevin Kho

03/30/2022, 1:45 PM
You can see more in this page. I agree this is a lot easier to do on the Prefect side than Cloudwatch
e

Eddie Atkinson

03/31/2022, 1:42 AM
My understanding was that Prefect doesn’t retain logs all that long (7 days or something from memory)? Has that changed? Ideally I want to be able to see logs for my tasks forever
k

Kevin Kho

03/31/2022, 2:22 AM
Yeah no we still don’t. No change. You’d need to send your logs to DataDog or a service like that
e

Eddie Atkinson

03/31/2022, 3:09 AM
I’ve just checked a task from October last year and it still has its logs. Is this because I am saving my logs to cloud watch? i.e. is Prefect fetching my logs from there and deleting them internally, or am I just lucky?
k

Kevin Kho

03/31/2022, 4:25 AM
Are you on Server or Cloud?
Enterprise level retention is 6 months
e

Eddie Atkinson

03/31/2022, 5:33 AM
Cloud
Interesting ok
10 Views