https://prefect.io logo
f

Francisco Tanudjaja

01/07/2021, 8:38 PM
Hi I'm new to prefect. I'm trying the docker image and ran into this issue
Copy code
$docker run -it prefecthq/prefect:latest bash

... within the container
root@8afe017ae8ef:/# prefect backend server
Backend switched to server
root@8afe017ae8ef:/# prefect server start
Exception caught; killing services (press ctrl-C to force)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/prefect/cli/server.py", line 347, in start
    ["docker-compose", "pull"], cwd=compose_dir_path, env=env
  File "/usr/local/lib/python3.7/subprocess.py", line 358, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/local/lib/python3.7/subprocess.py", line 339, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'docker-compose': 'docker-compose'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/prefect", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/prefect/cli/server.py", line 385, in start
    ["docker-compose", "down"], cwd=compose_dir_path, env=env
  File "/usr/local/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'docker-compose': 'docker-compose'
It's likely I'm missing something basic. Any tips appreciated. Thanks.
c

Chris White

01/07/2021, 8:54 PM
Hi @Francisco Tanudjaja - as the error message suggests, the CLI command
prefect server start
uses
docker-compose
under the hood. Running Docker within Docker is typically considered an anti-pattern and is not fully supported by docker
f

Francisco Tanudjaja

01/07/2021, 8:58 PM
Makes sense. I guess my question is how to use the docker image? The docs said
docker run -it prefecthq/prefect:latest
which seem to default to tini command with no additional info.
z

Zanie

01/07/2021, 9:05 PM
That docker image is typically for running flows in. If you’re trying to run an instance of Prefect Server, we’d recommend installing the
prefect
module locally then using the CLI to spin up the necessary docker containers.
upvote 1
f

Francisco Tanudjaja

01/07/2021, 9:17 PM
I see. We run a lot of our services on aws ECS ... I was hoping to use the docker image for doing something like that. So I guess you can't run prefect from a container since the underlying requires docker-compose?
z

Zanie

01/07/2021, 9:19 PM
Yeah so the difficulty of running it in a container based on a single image on ECS is that the server is composed of several services so each needs its own container.
You could use the docker compose file to understand how the services interact and run them all on ECS if you’d like, but that’s not something we have built out for you.
f

Francisco Tanudjaja

01/07/2021, 9:22 PM
Cool. Thanks for the info! I'll look into that.
z

Zanie

01/07/2021, 9:23 PM
There would still be modifications for it to work on your ECS environment I’m sure — let us know if you do end up doing this! It’s something I’d be happy to consider adding support for in the prefect/server repo. We also support deployments on K8s/EKS via Helm if that’s helpful.
2 Views