https://prefect.io logo
f

flavienbwk

10/09/2020, 3:02 PM
Hi ! I'm just starting off with Prefect and I was wondering if I can set up the Prefect server, Prefect UI and 1 Prefect worker directly in my own docker-compose file. Moreover, where do I find the ports to open and volumes to set up ?
n

nicholas

10/09/2020, 3:07 PM
Hi @flavienbwk -
prefect server start
acts as a wrapper for the pre-built
docker-compose.yaml
that orchestrates all the applications/nodes required to run the Prefect Server application. You can find that file here, and some configuration options here. Is there a reason you'd need to set up your own
docker-compose.yaml
to get started? I'd normally recommend against this since it adds a lot of friction for components that may be difficult to understand without a broader knowledge of the Prefect stack.
f

flavienbwk

10/09/2020, 3:11 PM
Thank you Nicholas. I work offline, so I wanted to have my own compose file to be able to do a
docker-compose up
with images coming from my own Docker registry.
n

nicholas

10/09/2020, 3:14 PM
Got it - if you're able to pull the images once with
prefect server start
, you can subsequently pass the
--skip-pull
arg to avoid having to re-pull images
f

flavienbwk

10/09/2020, 3:15 PM
I mean, it's a totally different server. I use docker pull / docker save on my internet computer and then docker load on my offline one. So I still can't use that method.
When you docker load an image, it gets the "source tag" (prefect/server:latest for example). So I could theorically use
prefect server start
with the
skip-pull
option. But it means I have manually to docker load my images in each of my offline computers ! I wanted to benefit from my registry to not force my users to manually pull the image from my registry and then ask them to re-tag it with the "source tag".
This causes another issue : Prefect doesn't seem to be "packageable" in a Kubernetes project, because it requires this
prefect server start
command that manages the containers to be launched.
c

Chris White

10/09/2020, 3:55 PM
Hi @flavienbwk - your frustrations are understandable, as Prefect Server is currently packaged / supported as a single user, single node deployment. There is community work on a helm chart for Server here: https://github.com/PrefectHQ/server/pull/57 if you’d like to give that a spin
f

flavienbwk

10/09/2020, 4:46 PM
Thank you @Chris White, but what alternative do I have ? Should I clone the git repo, edit your
docker-compose.yml
file and then
pip install ./prefect
it ? Will the containers launch with my docker images if I do that ?
c

Chris White

10/09/2020, 4:48 PM
That’s one way you could do it - you could also take the docker compose file standalone to avoid having to install
prefect
, and use
docker-compose up
directly with the appropriate environment variable settings
f

flavienbwk

10/09/2020, 4:48 PM
Great, so what does the
prefect server start
do in particular ? Just changing the env values ?
c

Chris White

10/09/2020, 4:50 PM
yup, it configures the environment in which
docker-compose up
is run, and responds to any passed user flags by altering the docker compose file. All logic for that command is here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/server.py
f

flavienbwk

10/09/2020, 4:50 PM
Thank you very much, I'll try it.
c

Chris White

10/09/2020, 4:51 PM
👍 👍
f

flavienbwk

10/09/2020, 5:28 PM
Hi again @Chris White , what about creating my tenant ? I get the following error :
Copy code
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: <http://localhost:4200/graphql>
I am able to access this URL from my computer.
c

Chris White

10/09/2020, 5:30 PM
I can’t help with your networking configuration, but the following will create a tenant for you on a machine that has API access:
Copy code
prefect backend server # from the terminal

# the following within a python session
>> from prefect import Client; c = Client()
>> client.create_tenant(name="default")
f

flavienbwk

10/09/2020, 5:31 PM
I mean, it's when running :
prefect server create-tenant --name default --slug default
c

Chris White

10/09/2020, 5:31 PM
ah i see…
did you run all the database migrations?
f

flavienbwk

10/09/2020, 5:32 PM
Hmm, not sure, I've just run the docker-compose.yml file. Is this operation automatic ? You can directly run the compose file by git cloning this project : https://github.com/flavienbwk/prefect-docker-compose I haven't changed anything, just added a
.env
file containing the default values
c

Chris White

10/09/2020, 5:34 PM
gotcha - it’s can be made automatic if you set
Copy code
PREFECT_SERVER_DB_CMD="prefect-server database upgrade -y"
in your environment prior to running `docker-compose up`; did you set that variable in your
.env
file?
1
f

flavienbwk

10/09/2020, 5:35 PM
I see ! It should be that, default is to "echo 'DATABASE MIGRATIONS SKIPPED"
c

Chris White

10/09/2020, 5:35 PM
ahh yes yes
f

flavienbwk

10/09/2020, 5:36 PM
Perfect, it works now! Thank you!
c

Chris White

10/09/2020, 5:37 PM
excellent - you’re welcome!