Owen Boyd
09/02/2025, 8:02 PMFlow run could not be submitted to infrastructure: The config profile (dev) could not be found
- I think that might be the aws config profile. Any advice?Marvin
09/02/2025, 8:02 PMMarvin
09/02/2025, 8:06 PMAWS_PROFILE=dev
) that doesn’t exist in the environment where the worker is running.
Two different “profiles” to consider:
- AWS profile: AWS_PROFILE
/ AWS_DEFAULT_PROFILE
used by boto3/botocore
- Prefect profile: PREFECT_PROFILE
used for Prefect settings
Given the timing (“could not be submitted to infrastructure”), this is almost certainly the AWS profile on the worker side.
What to check
1) Where is your worker running?
- Local machine: Make sure the profile exists locally and that the worker is started with the same environment
- Test:
echo $AWS_PROFILE
aws sts get-caller-identity
- If you rely on a named profile, start the worker with it:
AWS_PROFILE=dev prefect worker start --pool "<your-aws-pool>"
- ECS/EC2/Docker container: Do not use a named profile unless you’ve mounted ~/.aws
into the container. Prefer IAM roles. Unset profile env vars so the SDK uses the task/instance role:
unset AWS_PROFILE AWS_DEFAULT_PROFILE
Ensure the worker task/instance role has permissions to call ECS (e.g., ecs:RunTask, ecs:DescribeTasks, iam:PassRole as needed, logs:* for CloudWatch).
2) Inspect your work pool config for profile env vars
- Look for AWS_PROFILE
or AWS_DEFAULT_PROFILE
set in the pool’s base job template or in deployment overrides:
prefect work-pool inspect "<your-aws-pool>"
If you see a profile set there and you’re using IAM roles on the worker, remove it.
3) Sanity check it isn’t a Prefect profile
- If PREFECT_PROFILE=dev
is set in the worker environment but that Prefect profile isn’t present in the container/host, you’ll also see a similar message. Check:
echo $PREFECT_PROFILE
prefect profile ls
If dev
isn’t listed, either unset PREFECT_PROFILE
or create/use a profile that exists in that environment.
4) If you truly must use a named AWS profile inside a container
- Ensure the container has ~/.aws/config
and ~/.aws/credentials
with a [dev]
profile (e.g., mount your .aws
directory or bake credentials into the image), or set AWS_ACCESS_KEY_ID
, AWS_SECRET_ACCESS_KEY
, AWS_SESSION_TOKEN
, and AWS_DEFAULT_REGION
env vars in the worker container. IAM roles are strongly preferred.
Why this matters
- The credentials the worker uses are separate from the credentials your flow uses inside the ECS task. This error is occurring at submission time (worker side), so fix the worker’s AWS auth first. After that, ensure the task role used for your flow containers has the permissions your flow needs (e.g., S3 access).
Helpful references
- ECS work pool guide: Prefect 3: AWS ECS guide
- Similar report for context: GitHub issue: ProfileNotFound on ECS worker
If you can share:
- Which AWS work pool type (ECS, Batch?) you’re using
- Where the worker is running (local vs container vs ECS)
- Whether AWS_PROFILE
or PREFECT_PROFILE
is set anywhere (pool template, deployment, worker env)
…I can give you a pinpointed fix and the exact role policy/perm setup to use.