Hey team! I would like to ask again this here if a...
# ask-community
a
Hey team! I would like to ask again this here if anyone can help. Sometimes we get crashed flows in our infra with the following error:
Copy code
Flow run could not be submitted to infrastructure
We are running an Agent + ECS Tasks to run the flows. Does anyone have an idea of what the cause might be or what to do to improve this process? We would want to mitigate theses crashes as they have business impact.
1
r
Hey ! I’m not working on AWS but i’ve encountered that issue when the configuration for the job was not valid (unknown parameter etc...) Don’t you have more log on the issue ?
e
I'm also having the same issue
n
hi all! can those of you that have them please share non-sensitive portions of your agent logs? (the more you can provide, the more helpful it would be) I'm curious to see if you all are hitting the same problem Preferably in a snippet (Cmd+Shift+Enter on macos or + button) 🙂
e
@Robin Niel thanks for the inputs, I'll check if that may be the reason I'm having those issues
a
@NateI found the error trace on our DD, here it is:
Looks like we are creating one revision per flow run, which has a limit within AWS. Maybe we can switch to not create one per flow?
n
what version of
prefect-aws
are you on? we started being more careful about avoiding extra registrations in
0.4.10
here
a
Nice! I'm currently on 0.3.1, do I need to also change anything in Block definition for the ECS Task?
n
ah - my fault. I forgot we were talking about agents / blocks here - the change I linked above is for workers - sorry about that. I would recommend checking this out at some point in the near future, as our development / fixes have been / will be focused on workers going forward tldr: • we have a
MyInfraBlock.publish_as_work_pool()
to create the work pool from your infra block (applies to ECS) • pretty much just swap
prefect agent start
for
prefect worker start
👍 1
❤️ 1
a
Thank you very much @Nate, we will work on that and I'll give you feedback later,