https://prefect.io logo
#prefect-community
Title
# prefect-community
d

Darragh

06/22/2020, 9:01 PM
Has anyone had any success using the Docker Tasks on Fargate? I assumed it was possible, not sure if I’m barking up the wrong tree. I know there’s a variable in the CreateContainer Tasks to pass in the
docker_server_url
but I’ve no idea what that might be for this case. The error I get is:
DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))",)
Which traces back to line 302 in
containers.py
- there are some threads out there that suggest /`var/run` is the right answer, but no luck with it
k

Kyle Moon-Wright

06/22/2020, 9:22 PM
Hey @Darragh, I'm no expert on this topic, but I was able to gather some insight: The
docker_server_url
kwarg takes in an address of a Docker Daemon, but Fargate Tasks don't natively ship with one (though is is on their roadmap). With that in mind, exposing a daemon in some manner and calling out to it may be the best way to achieve what you're after. You may be better off using a Docker Agent on an EC2 instance or a Kubernetes Agent as detailed in this deployment recipe. Hope this helps!
Of course, others may have further insight on other approaches to this.
d

Darragh

06/22/2020, 9:36 PM
Thanks @Kyle Moon-Wright - could be a bit of an issue, were basing all of our executions off Fargate at the moment, so this would scupper the plans a fair bit. I would have thought it would be a reasonably common scenario, if anyone else has any experience of it?