https://prefect.io logo
Join Slack
Powered by
hello, I am running into an issue with an `ECS Age...
# ask-community
a
hello, I am running into an issue with an 
ECS Agent
. I get the below error on my flow run. I have checked, and found, the 
3.8.5
version (it was the docker container I built the flow in), but I can not determine where the 
3.7.10
version is. I looked in the 
EC2
instance that I set up and launched the agent from and it is at 
3.8.7
. Where else is Prefect running the flow that may have the 
3.7
Python version and is there any way around this other than upgrading that to 
3.8
?
✅ 1
Looks like it was in the 
prefecthq/prefect:latest
image
c
Makes sense — we have multiple images for different python versions that you can check out here: https://hub.docker.com/r/prefecthq/prefect
a
sweet
i do have one other question i just ran into
c
Fire away
a
i have a docker image in a private repo, and prefect can’t get to it because its unauthorized. IS there someway I can pass authorization in the flow configs, or do I need to set up ssh keys on the ECS agent
its a Github repo if that matters at all
my runconfig loks like
Copy code
RUN_CONFIG = ECSRun(labels=['s3-flow-storage'],
                      task_role_arn=TASK_ARN,
                      image=PREFECT_DATA_IMAGE,
                      memory=512,
                      cpu=256
                    )
c
Hmmm at the moment that needs to be set up on the ECS agent; the reason this isn’t available as a run config setting is essentially because the different agents all have drastically different auth patterns (e.g., docker agents use the docker daemon config, k8s agents use cluster-level config, etc.)
and all require some amount of configuration external to prefect
a
yeah i figured, just wanted to make sure i was on the right path
i seem to be climbing over one hurdle and running into the next one
i always feel like i am sooooo close to getting it fully working
lol you guys dont offer consulting to help get it set up do you
c
haha yea, we’re very aware of the DevOps requirements for getting a production ready setup and are actively brainstorming ideas for how to streamline the whole process. We actually do have an amazing partner that offers consulting services for Prefect! If you’re interested in learning more feel free to shoot me an email chris at prefect.io
a
i will do that because this is the last piece of my puzzle
c
awesome — and no pressure, but if you document some of the hurdles you experienced we will absolutely take them into consideration for documentation and feature enhancements
a
i started to
💯 1
alright, just shot it over
c
received - i’ll get you connected with the right folks 👍
a
thanks
thinking out loud as i do this, it would be pretty baller if you guys had example terraform templates for setting up different environments
c
In all seriousness we were discussing some feedback today requesting helm charts — I’ll link this to the team as another thing to consider 👍
a
very cool
so there isnt anything analogous to Airflow’s 
image_pull_secrets
? I just set up both SSH and docker login on the EC2 agent machine, but im still getting 
Unauthorized
. I think it is because the agent is spinning up another resource and not passing the credentials
looks like it would go in the 
containerDefinitions
json being passed to the task
would that be a custom task definition? and would that conflict with the ‘image’ key already being passed?
c
The kubernetes run config does have an image pull secret but it doesn’t appear the ECS config does; I’m not very fluent in AWS, I think for a higher quality response on the image pull secrets aspect you might want to open a GitHub issue (or start a new thread for others to chime in on) — I’m confident this is a popular requirement for folks
a
but would that be an example of what the 
custom task definition
argument is for?
c
yup exactly
3 Views
Open in Slack
PreviousNext
Powered by