https://prefect.io logo
m

Mattias Petter Johansson

09/14/2023, 6:48 PM
@Marvin what might be the reason for the ecs terraform template not picking up jobs? It says it is looking for work on queue default but nothing is happening. There is only one workpool in the cloud and running
prefect worker start --pool POOLNAME
locally works.
m

Marvin

09/14/2023, 6:49 PM
Ah, the existential dread of a malfunctioning ECS Terraform template. It's enough to make one wish they were a toaster. But alas, we must press on. There could be several reasons why your ECS Terraform template isn't picking up jobs. Here are a few possibilities: 1. API Endpoint Configuration: The Prefect API endpoint must be set in your agent's configuration. The ephemeral Prefect API doesn't work directly with ECS. You need to have a Prefect server or Prefect Cloud API endpoint set. 2. Prefect AWS Collection: The
prefect-aws
collection must be installed within the agent environment. You can do this with the command
pip install prefect-aws
. 3. Block Registration: The
ECSTask
and
AwsCredentials
blocks must be registered within the agent environment. You can do this with the command
prefect block register -m prefect_aws.ecs
. 4. Remote Storage Configuration: Local storage isn't supported for ECS tasks. You must configure remote storage. The most commonly used type of storage with
ECSTask
is S3. If you're using S3, ensure that
s3fs
is installed within your agent and flow run environment. 5. Template Variable Interpolation: If a template variable is the only value for a key in the
job_configuration
section, the key will be replaced with the value template variable. If a template variable is part of a string, there might be issues with text interpolation. If none of these seem to be the issue, it might be helpful to check the logs for any error messages or warnings. They might give you a clue as to what's going wrong. And remember, if all else fails, you can always consider the possibility that the universe is simply against you. It wouldn't be the first time.