https://prefect.io logo
Join Slack
Powered by
started playing with Prefect a bit on the weekend,...
# ask-community
t
started playing with Prefect a bit on the weekend, i have some questions about the "docker-compose" setup, e.g., how I would role my own. e.g., I don't have python on the host and would like to run it in containers a 100%. I also tried the suggested docker image, but that drops me into a shell. the docs are a bit opaque here. would anyone have some pointers? 🙂
Copy code
root@1864e79a297c:/# prefect server config
Traceback (most recent call last):
  File "/usr/local/bin/prefect", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/prefect/cli/server.py", line 394, in config_cmd
    subprocess.check_call(["docker-compose", "config"], cwd=compose_dir_path, env=env)
  File "/usr/local/lib/python3.7/subprocess.py", line 358, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/local/lib/python3.7/subprocess.py", line 339, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'docker-compose': 'docker-compose'
j
Hi Till, The easiest way to spin up prefect in docker is to use docker compose. Most users spin it up by also installing 
prefect
on the node and using 
prefect server start
, but you can avoid needing prefect (and only require 
docker-compose
) by pre-generating the 
docker-compose.yaml
file elsewhere.
Copy code
# Generate the docker-compose.yaml anywhere you have `prefect` installed
$ prefect server config > docker-compose.yaml

# Copy the docker-compose.yaml to the node you want prefect server to run on
# then run the following in the same directory as the docker-compose.yaml
$ docker-compose up --detach
Note that if you don't use our CLI (
prefect server start
), you'll also need to manually create a new tenant after your first startup (to populate the database). This can be done from any node, and only needs to be run once:
Copy code
prefect server create-tenant --name default --slug default
t
@Jim Crist-Harif yeah, been trying to avoid that a bit. but it seems i need python and prefect somewhere to generate it initially?
i could PR the output some place (to docs), if that is okay? i see no reason not to show it
🙂
j
Since you need prefect somewhere anyway to run an agent/write user code, I don't fully see what the issue is. The original 
docker-compose.yaml
file is here for reference: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/docker-compose.yml. The CLI sets (or templates in) the required configuration environment variables, so it's not really meant to be used on its own.
t
ok
my end goal would be a small containerized setup which I can give to my colleagues. to help with eval, since we are not "python programmers" we prefer everything containerized. (also for later on). we just run docker nodes, with nothing installed, etc.. i am reading another thread by @Florian Kühnlenz — I guess I am as confused as he is. but I'll see what you wrote to him and see if can make it work.
f
@Jim Crist-Harif For me the biggest problem was discovery and clarity in the documentation. I think it would be great if a section could be added explaining that prefect server config will create a docker-compose file and how the tenant creation works. Essentially a step by step on how to do what the scripts do. The other problem for me, even as a python programmer, was the need to install prefect as a package. I do not want to install it in my system python. For projects we manage environments via poetry. This in turn makes it difficult to use the scripts you provide easily. For example because docker-compose wants sudo rights on my machine. I did not manage to run the poetry env in a way to make this possible. I understand that the scrips should provide maximum convenience but for me they did the opposite.
t
Maybe to add (not to bash): I am not sure what the docker image currently does. Or let me put it this way: what it can be used for and then why it's mentioned as a way to run prefect. The 
docker run
drops me into a "shell", but then it needs 
docker-compose
which is not included in the image (probably for good reason). 😄 I created most of what I needed in a 
pipenv
to limit the impact of a system-wide install.
f
I understood the docker images can be used as worker agents, but could be totally off.
👀 1
4 Views
Open in Slack
PreviousNext
Powered by