<@ULVA73B9P> I created a new workpool on aws to te...
# ask-marvin
o
@Marvin I created a new workpool on aws to test out higher memory limits. I updated my dev deployment, tried to run a flow and got the following error:
Flow run could not be submitted to infrastructure: The config profile (dev) could not be found
- I think that might be the aws config profile. Any advice?
m
thought for 275.5 seconds
You’re right to suspect an AWS profile. That error usually comes from botocore when the worker is trying to submit your flow to ECS (or other AWS service) using a named profile (e.g.,
AWS_PROFILE=dev
) that doesn’t exist in the environment where the worker is running. Two different “profiles” to consider: - AWS profile:
AWS_PROFILE
/
AWS_DEFAULT_PROFILE
used by boto3/botocore - Prefect profile:
PREFECT_PROFILE
used for Prefect settings Given the timing (“could not be submitted to infrastructure”), this is almost certainly the AWS profile on the worker side. What to check 1) Where is your worker running? - Local machine: Make sure the profile exists locally and that the worker is started with the same environment - Test:
Copy code
echo $AWS_PROFILE
    aws sts get-caller-identity
- If you rely on a named profile, start the worker with it:
Copy code
AWS_PROFILE=dev prefect worker start --pool "<your-aws-pool>"
- ECS/EC2/Docker container: Do not use a named profile unless you’ve mounted
~/.aws
into the container. Prefer IAM roles. Unset profile env vars so the SDK uses the task/instance role:
Copy code
unset AWS_PROFILE AWS_DEFAULT_PROFILE
Ensure the worker task/instance role has permissions to call ECS (e.g., ecs:RunTask, ecs:DescribeTasks, iam:PassRole as needed, logs:* for CloudWatch). 2) Inspect your work pool config for profile env vars - Look for
AWS_PROFILE
or
AWS_DEFAULT_PROFILE
set in the pool’s base job template or in deployment overrides:
Copy code
prefect work-pool inspect "<your-aws-pool>"
If you see a profile set there and you’re using IAM roles on the worker, remove it. 3) Sanity check it isn’t a Prefect profile - If
PREFECT_PROFILE=dev
is set in the worker environment but that Prefect profile isn’t present in the container/host, you’ll also see a similar message. Check:
Copy code
echo $PREFECT_PROFILE
  prefect profile ls
If
dev
isn’t listed, either unset
PREFECT_PROFILE
or create/use a profile that exists in that environment. 4) If you truly must use a named AWS profile inside a container - Ensure the container has
~/.aws/config
and
~/.aws/credentials
with a
[dev]
profile (e.g., mount your
.aws
directory or bake credentials into the image), or set
AWS_ACCESS_KEY_ID
,
AWS_SECRET_ACCESS_KEY
,
AWS_SESSION_TOKEN
, and
AWS_DEFAULT_REGION
env vars in the worker container. IAM roles are strongly preferred. Why this matters - The credentials the worker uses are separate from the credentials your flow uses inside the ECS task. This error is occurring at submission time (worker side), so fix the worker’s AWS auth first. After that, ensure the task role used for your flow containers has the permissions your flow needs (e.g., S3 access). Helpful references - ECS work pool guide: Prefect 3: AWS ECS guide - Similar report for context: GitHub issue: ProfileNotFound on ECS worker If you can share: - Which AWS work pool type (ECS, Batch?) you’re using - Where the worker is running (local vs container vs ECS) - Whether
AWS_PROFILE
or
PREFECT_PROFILE
is set anywhere (pool template, deployment, worker env) …I can give you a pinpointed fix and the exact role policy/perm setup to use.