https://prefect.io logo
Join Slack
Powered by
Hi, I built a jenkins pipeline, to deploy flows au...
# prefect-cloud
s
Hi, I built a jenkins pipeline, to deploy flows automatically every time a PR is merged in my github repo. but for some reason is failing: I am using a 
docker-git
recipe, and I am running these commands:
Copy code
prefect cloud login -k {key} -w {workspace}
prefect project init --recipe docker-git -f image_name=some-docker-image -f tag=latest -f repository={repository} -f branch=master
prefect deploy {flow_path}.py:{access_method} -n {flow_name} -p pool
Output:
Copy code
Warning: extra fields provided for 'docker-git' recipe: 'repository, branch'
12:52:38  ---------------
12:52:38  Created project in /home/ec2-user/workspace/_Team_automation-flows-v2_master 
12:52:38  with the following new files:
12:52:38  deployment.yaml
12:52:38  prefect.yaml
12:52:38  .prefect/
12:52:41  An exception occurred.
12:52:41  Traceback (most recent call last):
12:52:41    File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
12:52:41      httplib_response = self._make_request(
12:52:41    File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
12:52:41      conn.request(method, url, **httplib_request_kw)
12:52:41    File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 244, in request
12:52:41      super(HTTPConnection, self).request(method, url, body=body, headers=headers)
12:52:41    File "/usr/local/lib/python3.8/http/client.py", line 1252, in request
12:52:41      self._send_request(method, url, body, headers, encode_chunked)
12:52:41    File "/usr/local/lib/python3.8/http/client.py", line 1298, in _send_request
12:52:41      self.endheaders(body, encode_chunked=encode_chunked)
12:52:41    File "/usr/local/lib/python3.8/http/client.py", line 1247, in endheaders
12:52:41      self._send_output(message_body, encode_chunked=encode_chunked)
12:52:41    File "/usr/local/lib/python3.8/http/client.py", line 1007, in _send_output
12:52:41      self.send(msg)
12:52:41    File "/usr/local/lib/python3.8/http/client.py", line 947, in send
12:52:41      self.connect()
12:52:41    File "/usr/local/lib/python3.8/site-packages/docker/transport/unixconn.py", line 27, in connect
12:52:41      sock.connect(self.unix_socket)
12:52:41  FileNotFoundError: [Errno 2] No such file or directory
Copy code
File "/usr/local/lib/python3.8/contextlib.py", line 113, in __enter__
12:52:41      return next(self.gen)
12:52:41    File "/usr/local/lib/python3.8/site-packages/prefect/docker.py", line 102, in docker_client
12:52:41      client = docker.DockerClient.from_env()
12:52:41    File "/usr/local/lib/python3.8/site-packages/docker/client.py", line 96, in from_env
12:52:41      return cls(
12:52:41    File "/usr/local/lib/python3.8/site-packages/docker/client.py", line 45, in __init__
12:52:41      self.api = APIClient(*args, **kwargs)
12:52:41    File "/usr/local/lib/python3.8/site-packages/docker/api/client.py", line 197, in __init__
12:52:41      self._version = self._retrieve_server_version()
12:52:41    File "/usr/local/lib/python3.8/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
12:52:41      raise DockerException(
12:52:41  docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
n
do you have docker running on your EC2?
s
when I run it on my local, I have the same issue
and yes, docker is running
n
hmm, i have only ever seen this error when docker is not running (i.e. docker client cannot find / connect to the unix socket) or docker is misconfigured how did you install docker where you're seeing this error?
s
I ’ve downloaded it from docker web page
You are right, something is wrong with docker
trying to fix
I have another error now
Copy code
Traceback (most recent call last):
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
16:47:58      return fn(*args, **kwargs)
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 260, in coroutine_wrapper
16:47:58      return call()
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 245, in __call__
16:47:58      return self.result()
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
16:47:58      return self.future.result(timeout=timeout)
16:47:58    File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 437, in result
16:47:58      return self.__get_result()
16:47:58    File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
16:47:58      raise self._exception
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
16:47:58      result = await coro
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/cli/deploy.py", line 241, in deploy
16:47:58      await _run_single_deploy(
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/cli/deploy.py", line 394, in _run_single_deploy
16:47:58      step_outputs.update(await run_step(step))
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/projects/steps/core.py", line 78, in run_step
16:47:58      return await from_async.call_soon_in_new_thread(
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 181, in aresult
16:47:58      return await asyncio.wrap_future(self.future)
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 194, in _run_sync
16:47:58      result = self.fn(*self.args, **self.kwargs)
16:47:58    File "/usr/local/lib/python3.8/site-packages/prefect_docker/projects/steps.py", line 143, in build_docker_image
16:47:58      image.tag(repository=image_name, tag=tag)
16:47:58    File "/usr/local/lib/python3.8/site-packages/docker/models/images.py", line 141, in tag
16:47:58      return self.client.api.tag(self.id, repository, tag=tag, **kwargs)
16:47:58    File "/usr/local/lib/python3.8/site-packages/docker/utils/decorators.py", line 19, in wrapped
16:47:58      return f(self, resource_id, *args, **kwargs)
16:47:58    File "/usr/local/lib/python3.8/site-packages/docker/api/image.py", line 571, in tag
16:47:58      self._raise_for_status(res)
16:47:58    File "/usr/local/lib/python3.8/site-packages/docker/api/client.py", line 270, in _raise_for_status
16:47:58      raise create_api_error_from_http_exception(e) from e
16:47:58    File "/usr/local/lib/python3.8/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
16:47:58      raise cls(e, response=response, explanation=explanation) from e
16:47:58  docker.errors.APIError: 500 Server Error for <http+docker://localhost/v1.41/images/sha256:99ab7fb42b5b778ede1e466eb11dab7c7ebe366853e181fa2baef522793468a0/tag?tag=latest&repo=https%3A%2F%2Fdocker.util.pages%2Fdata%2Fautomation-v2&force=0>: Internal Server Error ("invalid reference format")
n
how have you set your tag for your image?
s
Copy code
prefect project init --recipe docker-git -f image_name=<https://docker.util.pages/data/automation-v2> -f tag=latest -f repository=<https://github.dev.pages/data/automation-flows-v2.git> -f branch=master
n
hmm, are you able to build a docker image otherwise on that machine?
s
Yes
I’ve just realized what was causing the issue (however the error should be easier to understand). I was providing the 
image_name
with 
https
protocol within.
Open in Slack
PreviousNext
Powered by