I am running flows on prefect 1.0 with an ECS Agen...
# prefect-community
I am running flows on prefect 1.0 with an ECS Agent. The flow is successfully scheduled/submitted then gets stuck in that state. I looked on ECS and it looks like the container starts then immediately dies. There aren’t any associated logs that I can find. Any recommendations on how to debug this? EDIT - ended up being a 1.0 agent/2.0flow compatibility issue. For brevity the ECS container stop reason was
essential container in task exited
with no generated logs. The entrance command
["/bin/sh", "-c", "prefect execute flow-run"]
was failing because (I think) prefect 2.0 doesn’t have an
command. I resolved this by pinning my prefect dependency to
pip install "prefect[aws, gitlab]==1.3.0"
yes, You should go look at the log of your container in AWS. Sometimes there is more informations like a grant problem on the docker images you're using.
Thanks for the response @Benjamin.bgx - the container doesn’t generate any logs that I can find from within the ECS task log area or on cloud watch. Are there other locations you look?
I think it means the task did not start. You should see some information in the task specific information panel. Let me see if I can find some screenshot to show you.
I had this exact same issue. First time I encountered it, it was due to a mismatch in the docker architecture. Are you building the images on an M1 machine?
@Tomas Knoetze I’ve already fought that fight as well haha - the containers are build on a gitlab runner that is
so no M1 issues. @Benjamin.bgx I think you are correct; I think the reason it was exiting immediately is because the entrance point to the container is
/bin/sh -c prefect execute flow-run
. I didn’t have prefect 1.0 pinned when building the container and
doesn’t appear to be a command in 2.0. I pinned the dependency to
"prefect[aws, gitlab]==1.3.0"
and now the flow will run