:wave: I'm running into issues setting up Fargate ...
# ask-community
c
šŸ‘‹ I'm running into issues setting up Fargate tasks. The agent successfully launches the ECS task, but the task fails at the validation stage after it loads the flow from S3. It thinks the task and container definitions are different, but neither has changed since I registered the flow and the agent scheduled it. I wonder if it's because the flow is using S3 storage? I didn't use Docker because I couldn't figure out how to use it with ECR and provide credentials. Thoughts?
Copy code
2020-11-06T07:56:19.166-05:00

Copy
	containerDefinition.flow-container.command -> Given: ['/bin/sh', '-c', "python -c 'import prefect; prefect.environments.execution.load_and_run_flow()'"], Expected: ['/bin/sh', '-c', 'prefect execute flow-run']
containerDefinition.flow-container.command -> Given: ['/bin/sh', '-c', "python -c 'import prefect; prefect.environments.execution.load_and_run_flow()'"], Expected: ['/bin/sh', '-c', 'prefect execute flow-run']
a
Hi @Carlo, which version of Prefect are you using? I faced a similar issue with Prefect version < 0.13.13, but it should have been solved in 0.13.13
c
I think I'm on an older version than that, let me upgrade. šŸ™ Thank you
a
Let me know how it goes
c
I set everything to
0.13.13
but the error still persists. Are you using s3 storage?
a
Nope, I’m using Docker/ECR
However, you error seems to be related to the comparison between the existing task definition and the new task definition. Maybe @Spencer can help
s
prefect execute flow-run
would be used by the Agent
The task family that the Fargate Agent uses should be different from the FargateTaskEnvironment task family
c
I'm not setting a command explicitly in either the flow fargate task env or the agent
s
That's correct
c
I do have family set on the flow fargate task env, but not setting one on the agent
s
So, the Agent will create an ECS Task Definition which when executed will create another Task Definition to run the flow. My intuition is that these are referencing the same definition for some reason.
c
oh, I think it might be because I have taskDefinition and family set to the same value in the fargate task env
let me change this
s
Well, in the FargateTaskEnvironment, the
taskDefinition
and
family
are typically the same. I think those values may be the same as the flow-run
taskDefinition
that the Agent is creating. You can look in the AWS ECS console to see the existing
taskDefinition
c
This solved it, different errors now, but I think I can fix these
s
Oh ok. It seems that there could be a really odd scenario where the
taskDefinition
here is named the same as a flow's flow-run
taskDefinition
would be by the Agent. This probably won't happen if you're not using
enable_task_revisions
since you'll get a fairly unique name, otherwise you could have a flow has a name that would collide. ( https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/fargate/agent.py#L478-L487 )
c
I will do some more research and come back
I think it's because I had
flow.name
fargateTaskEnvironment.family
fargateTaskEnvironment.taskDefinition
all set to the same value
šŸ‘ 1
I was able to get my flow working. Thanks for the help, it really unblocked me.