Hello,
Anyone seen this `crashed` in prefect UI ha...
# best-practicesj
Jafar A
03/15/2023, 9:33 PMHello,
Anyone seen this
crashedFlow run infrastructure exited with non-zero status code 1.17:26:06.079 | DEBUG | prefect.agent - Checking for cancelled flow runs...17:26:07.931 | INFO | prefect.infrastructure.ecs-task - ECSTask 'electric-guan': Status is RUNNING.17:26:09.893 | DEBUG | prefect.agent - Checking for scheduled flow runs...17:26:12.938 | INFO | prefect.infrastructure.ecs-task - ECSTask 'electric-guan': Running command 'python -m prefect.engine' in container 'prefect' (<http://880954911558.dkr.ecr.us-west-2.amazonaws.com/prefect-docker-creditfurnishing:latest|880954911558.dkr.ecr.us-west-2.amazonaws.com/prefect-docker-creditfurnishing:latest>)...17:26:12.971 | INFO | prefect.infrastructure.ecs-task - ECSTask 'electric-guan': Streaming output from container 'prefect'...17:26:13.023 | INFO | prefect.agent - Completed submission of flow run '57ccf19d-e50c-469e-9697-5a5b46011ba9'17:26:13.059 | INFO | prefect.infrastructure.ecs-task - ECSTask 'electric-guan': Status is DEPROVISIONING.exec /usr/bin/tini: exec format error17:26:16.249 | DEBUG | prefect.agent - Checking for cancelled flow runs...17:26:21.936 | DEBUG | prefect.agent - Checking for scheduled flow runs...17:26:24.004 | DEBUG | prefect.agent - Checking for cancelled flow runs...17:26:30.948 | INFO | prefect.infrastructure.ecs-task - ECSTask 'electric-guan': Status is STOPPED.17:26:31.550 | WARNING | prefect.infrastructure.ecs-task - ECSTask 'electric-guan': Container 'prefect' exited with non-zero exit code 1.17:26:31.708 | DEBUG | prefect.agent - Checking for cancelled flow runs...17:26:31.726 | INFO | prefect.agent - Reported flow run '57ccf19d-e50c-469e-9697-5a5b46011ba9' as crashed: Flow run infrastructure exited with non-zero status code 1.17:26:32.807 | DEBUG | prefect.agent - Checking for scheduled flow runs...17:26:41.023 | DEBUG | prefect.agent - Checking for cancelled flow runs...17:26:42.106 | DEBUG | prefect.agent - Checking for scheduled flow runs...z
Zanie
03/15/2023, 10:44 PM
It’s possible you are encountering https://github.com/PrefectHQ/prefect-aws/pull/218
Zanie
03/15/2023, 10:45 PM
Actually nevermind this looks like a real error. See
exec /usr/bin/tini: exec format error
Zanie
03/15/2023, 10:45 PM
Looks like a problem with your image?
j
Jafar A
03/16/2023, 2:33 PM@Zanie it appears so however, when I try same deployment with docker container block (instead of ECS block) it pulls the image from ecr and runs fine!
looks like this is ECS related issue?
Jafar A
03/16/2023, 3:06 PMI think some are pointing to ARM processor that my local machine is building containers and looks like Fargate is using x86 so will try fixing this discrepancy and see.
z
Zanie
03/16/2023, 3:11 PM
Ah that makes sense
j
Jafar A
03/16/2023, 3:23 PM@Zanie quick question , I thin I need to add below json into the task definition which I manually created a new revision in ECS task definition with ARM (rev 8).
Now when I run prefect job on UI, it'll skip my revision and jumps to create a new revision (9) that doesn't have ARM64.
do you know if there is a way to use my revision or to permanently default to ARM64?
the other way is to insert this into yaml file but deployment will revert it back (under DO NOT EDIT UNDER THIS LINE).
Any advice?
"requiresCompatibilities": [ "FARGATE" ],"cpu": "256","memory": "512","runtimePlatform": {"operatingSystemFamily": "LINUX","cpuArchitecture": "ARM64"},z
Zanie
03/16/2023, 3:36 PM
Do you need to update the task definition arn you’re linking to in your
ECSTaskj
Jafar A
03/16/2023, 3:39 PMI just linked my revision to the block and boom works great thanks.
Jafar A
03/16/2023, 3:39 PMdo you know how to stop auto-creating all those revisions? I don't know if this is doing any good but happy to hear thoughts
z
Zanie
03/16/2023, 3:49 PM
We only create a revision if you’re passing options that require us to create a new revision 😄
Zanie
03/16/2023, 3:50 PM
6 Views