https://prefect.io logo
Join Slack
Powered by
Hi all. I want to use dependency between docker im...
# ask-community
v
Hi all. I want to use dependency between docker images. For example, I have 1 container 
myapp
and another container which is actually the prefect 
agent
that will be running the flows. I figured out the 
agent
needs to have the code that is there in 
myapp
. So, I set up the flow like below,
Copy code
flow.storage = Docker(dockerfile='path_of_Dockerfile_inside_myapp')  #this will ensure that Dockerfile I used to build myapp will be used to build agent
flow.storage.build() #this step fails with the eroor: FileNotFound. I looked at the prefect code in the github repo and this should work, but fails
flow.register(labels=['some_label'])
Is there a gap in my understanding of how the Docker storage works?
j
Hey @Vikram Iyer can you verify that the place you are calling this build from has access to the Dockerfile in the same path that you pass to 
dockerfile=…
Also as a side note, you might not need to build the storage directly because 
flow.register
has a 
build
kwarg which defaults to 
True
v
The place I am calling from is: 
/app/somedir/code.py
and The place where the Dockerfile is, is 
/app/Dockerfile
. Ideally the parent dir so should be accessible. What do you think?
j
Could you post the full traceback that you get when you run 
.build
?
v
Copy code
myapp  | /usr/local/lib/python3.7/site-packages/prefect/client/client.py:619: UserWarning: No result handler was specified on your Flow. Cloud features such as input caching and resuming task runs from failure may not work properly.
myapp  |   "No result handler was specified on your Flow. Cloud features such as "
myapp  | /usr/local/lib/python3.7/site-packages/prefect/core/flow.py:1371: UserWarning: A flow with the same name is already contained in storage; if you changed your Flow since the last build, you might experience unexpected issues and should re-create your storage object.
myapp  |   "A flow with the same name is already contained in storage; if you "
myapp  | [2020-07-02 00:43:18 +0530] [50] [ERROR] Error handling request /api/v1/metric/add/
myapp  | Traceback (most recent call last):
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
myapp  |     chunked=chunked,
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
myapp  |     conn.request(method, url, **httplib_request_kw)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1252, in request
myapp  |     self._send_request(method, url, body, headers, encode_chunked)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1298, in _send_request
myapp  |     self.endheaders(body, encode_chunked=encode_chunked)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1247, in endheaders
myapp  |     self._send_output(message_body, encode_chunked=encode_chunked)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1026, in _send_output
myapp  |     self.send(msg)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 966, in send
myapp  |     self.connect()
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
myapp  |     sock.connect(self.unix_socket)
myapp  | FileNotFoundError: [Errno 2] No such file or directory
myapp  |
myapp  | During handling of the above exception, another exception occurred:
myapp  |
myapp  | Traceback (most recent call last):
myapp  |   File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
myapp  |     timeout=timeout
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
myapp  |     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 403, in increment
myapp  |     raise six.reraise(type(error), error, _stacktrace)
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
myapp  |     raise value.with_traceback(tb)
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
myapp  |     chunked=chunked,
myapp  |   File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
myapp  |     conn.request(method, url, **httplib_request_kw)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1252, in request
myapp  |     self._send_request(method, url, body, headers, encode_chunked)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1298, in _send_request
myapp  |     self.endheaders(body, encode_chunked=encode_chunked)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1247, in endheaders
myapp  |     self._send_output(message_body, encode_chunked=encode_chunked)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 1026, in _send_output
myapp  |     self.send(msg)
myapp  |   File "/usr/local/lib/python3.7/http/client.py", line 966, in send
myapp  |     self.connect()
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
myapp  |     sock.connect(self.unix_socket)
myapp  | urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
myapp  |
myapp  | During handling of the above exception, another exception occurred:
myapp  |
myapp  | Traceback (most recent call last):
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 207, in _retrieve_server_version
myapp  |     return self.version(api_version=False)["ApiVersion"]
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/api/daemon.py", line 181, in version
myapp  |     return self._result(self._get(url), json=True)
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
myapp  |     return f(self, *args, **kwargs)
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 230, in _get
myapp  |     return self.get(url, **self._set_request_timeout(kwargs))
myapp  |   File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 543, in get
myapp  |     return self.request('GET', url, **kwargs)
myapp  |   File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
myapp  |     resp = self.send(prep, **send_kwargs)
myapp  |   File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
myapp  |     r = adapter.send(request, **kwargs)
myapp  |   File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
myapp  |     raise ConnectionError(err, request=request)
myapp  | requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
myapp  |
myapp  | During handling of the above exception, another exception occurred:
myapp  |
myapp  | Traceback (most recent call last):
myapp  |   File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 134, in handle
myapp  |     self.handle_request(listener, req, client, addr)
myapp  |   File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 175, in handle_request
myapp  |     respiter = self.wsgi(environ, resp.start_response)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2464, in __call__
myapp  |     return self.wsgi_app(environ, start_response)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2450, in wsgi_app
myapp  |     response = self.handle_exception(e)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1867, in handle_exception
myapp  |     reraise(exc_type, exc_value, tb)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
myapp  |     raise value
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
myapp  |     response = self.full_dispatch_request()
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
myapp  |     rv = self.handle_user_exception(e)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
myapp  |     reraise(exc_type, exc_value, tb)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
myapp  |     raise value
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
myapp  |     rv = self.dispatch_request()
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
myapp  |     return self.view_functions[rule.endpoint](**req.view_args)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask_classful.py", line 268, in proxy
myapp  |     response = view(**request.view_args)
myapp  |   File "/usr/local/lib/python3.7/site-packages/flask_classful.py", line 239, in inner
myapp  |     return fn(*args, **kwargs)
myapp  |   File "/app/outlier_detection/api/v1/metric.py", line 35, in add
myapp  |     response = runtime_flow(data_dict)
myapp  |   File "/app/outlier_detection/flows/runtime_flow.py", line 68, in runtime_flow
myapp  |     flow.register(labels=['l1'])
myapp  |   File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1527, in register
myapp  |     no_url=no_url,
myapp  |   File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 662, in register
myapp  |     serialized_flow = flow.serialize(build=build)  # type: Any
myapp  |   File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1375, in serialize
myapp  |     storage = self.storage.build()  # type: Optional[Storage]
myapp  |   File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 304, in build
myapp  |     self._build_image(push=push)
myapp  |   File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 341, in _build_image
myapp  |     client = self._get_client()
myapp  |   File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 531, in _get_client
myapp  |     base_url=self.base_url, version="auto", tls=self.tls_config
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 190, in __init__
myapp  |     self._version = self._retrieve_server_version()
myapp  |   File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 215, in _retrieve_server_version
myapp  |     'Error while fetching server API version: {0}'.format(e)
myapp  | docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
myapp  | 192.168.192.1 - - [02/Jul/2020:00:43:18 +0530] "POST /api/v1/metric/add/ HTTP/1.1" 500 0 "-" "-" in 10929µs
j
Oh, you’re attempting to register the flow using Docker inside of a Docker container and it cannot find a daemon to connect to
Docker
storage has a 
base_url
kwarg in which you can pass in the url of a Docker daemon in order to build the storage.
Optionally you could also mount your local daemon to your 
myapp
container to achieve this https://medium.com/better-programming/about-var-run-docker-sock-3bfd276e12fd
Is there a reason why you are registering your flow inside of a container instead of on your local machine?
v
Oh! Cool, I will try using the base_url. Just curious, is registering a flow from within a container using Docker() not recommended.
j
It’s not that it isn’t recommended but it requires some extra setup 🙂 and if you already have Docker running on your machine it would make more sense to register there
v
Short answer: To be able to deploy anywhere. Long answer: So, I have this huge docker-compose that has all myapp + other services along with prefect ones (ui, scheduler, apollo, etc). So this myapp service will accept the flow details and register them.
I hope this makes sense, I will try drawing an HLD to explain if not.
Something like this!
j
Yeah I follow what you’re attempting. In this case where you’re trying to build an image from within a container you should do one of the options from above (mounting daemon / providing 
base_url
/ etc.). Otherwise you could look into using a different storage option in conjunction with your docker-based agent https://docs.prefect.io/orchestration/execution/storage_options.html#non-docker-storage-for-containerized-environments
v
Sure. The base_url should be something like hostip:docker_daemon_port right?
j
Yep
v
Hey @josh. I thought I'd let you know. I mounted the same image for both myapp and the agent and things worked just fine. Did not have to use Docker() since I did not want a new container to be up, rather wanted the agent container to take care of it. Thanks for all the help 🙂
👍 2
m
@Vikram Iyer I'm trying to do something similar as you workflow, but I have some doubts about it. How did you register the image on the agent? Every agent is registered for a single flow or could be reutilized for different ones?. I have a docker-compose with all the stack for prefect (serving the ui and apollo with traefik 2), and I would like to create differents flows but dettached for that docker-compose stack.
v
Hey @Martin. I built the image for myapp and pointed the agent container to refer to the same image. This way both myapp and agent containers have the same image and hence the code within. If the current directory on your host has the code, you can use 
build: .
in docker-compose to say that both myapp and agent should have the current dir built into them. As a final step, make sure to mount the code directory inside both of them. Does that answer your question?
m
Yes for sure! thanks!. So one agent per flow no?
Do you have a docker-compose example with that flow?
v
Not actually. There is an agent running that can accept any flow. Yep, let me share a snapshot of the compose file.
You might have the original docker compose. To that you can add below 2 services, 1 is the app and the other is agent like so:
Copy code
myapp:
    build: .
    command: "command to run your application/service"
    networks:
      - prefect-server
    environment:
       PREFECT__BACKEND: server
       PREFECT__SERVER__ENDPOINT: <http://apollo:4200>
       PREFECT__SERVER__GRAPHQL_URL: <http://apollo:4200/graphql>

agent:
    build: .
    command: "prefect agent start --label anylabel"
    networks:
      - prefect-server
    environment:
      PREFECT__SERVER__GRAPHQL_URL: <http://apollo:4200/graphql>
      PREFECT__SERVER__ENDPOINT: <http://apollo:4200>
    restart: "always"
m
ohh nice! thanks!
2 Views
Open in Slack
PreviousNext
Powered by