flavienbwk

    flavienbwk

    1 year ago
    Hi ! I'm just starting off with Prefect and I was wondering if I can set up the Prefect server, Prefect UI and 1 Prefect worker directly in my own docker-compose file. Moreover, where do I find the ports to open and volumes to set up ?
    nicholas

    nicholas

    1 year ago
    Hi @flavienbwk -
    prefect server start
    acts as a wrapper for the pre-built
    docker-compose.yaml
    that orchestrates all the applications/nodes required to run the Prefect Server application. You can find that file here, and some configuration options here. Is there a reason you'd need to set up your own
    docker-compose.yaml
    to get started? I'd normally recommend against this since it adds a lot of friction for components that may be difficult to understand without a broader knowledge of the Prefect stack.
    flavienbwk

    flavienbwk

    1 year ago
    Thank you Nicholas. I work offline, so I wanted to have my own compose file to be able to do a
    docker-compose up
    with images coming from my own Docker registry.
    nicholas

    nicholas

    1 year ago
    Got it - if you're able to pull the images once with
    prefect server start
    , you can subsequently pass the
    --skip-pull
    arg to avoid having to re-pull images
    flavienbwk

    flavienbwk

    1 year ago
    I mean, it's a totally different server. I use docker pull / docker save on my internet computer and then docker load on my offline one. So I still can't use that method.
    When you docker load an image, it gets the "source tag" (prefect/server:latest for example). So I could theorically use
    prefect server start
    with the
    skip-pull
    option. But it means I have manually to docker load my images in each of my offline computers ! I wanted to benefit from my registry to not force my users to manually pull the image from my registry and then ask them to re-tag it with the "source tag".
    This causes another issue : Prefect doesn't seem to be "packageable" in a Kubernetes project, because it requires this
    prefect server start
    command that manages the containers to be launched.
    Chris White

    Chris White

    1 year ago
    Hi @flavienbwk - your frustrations are understandable, as Prefect Server is currently packaged / supported as a single user, single node deployment. There is community work on a helm chart for Server here: https://github.com/PrefectHQ/server/pull/57 if you’d like to give that a spin
    flavienbwk

    flavienbwk

    1 year ago
    Thank you @Chris White, but what alternative do I have ? Should I clone the git repo, edit your
    docker-compose.yml
    file and then
    pip install ./prefect
    it ? Will the containers launch with my docker images if I do that ?
    Chris White

    Chris White

    1 year ago
    That’s one way you could do it - you could also take the docker compose file standalone to avoid having to install
    prefect
    , and use
    docker-compose up
    directly with the appropriate environment variable settings
    flavienbwk

    flavienbwk

    1 year ago
    Great, so what does the
    prefect server start
    do in particular ? Just changing the env values ?
    Chris White

    Chris White

    1 year ago
    yup, it configures the environment in which
    docker-compose up
    is run, and responds to any passed user flags by altering the docker compose file. All logic for that command is here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/server.py
    flavienbwk

    flavienbwk

    1 year ago
    Thank you very much, I'll try it.
    Chris White

    Chris White

    1 year ago
    👍 👍
    flavienbwk

    flavienbwk

    1 year ago
    Hi again @Chris White , what about creating my tenant ? I get the following error :
    requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: <http://localhost:4200/graphql>
    I am able to access this URL from my computer.
    Chris White

    Chris White

    1 year ago
    I can’t help with your networking configuration, but the following will create a tenant for you on a machine that has API access:
    prefect backend server # from the terminal
    
    # the following within a python session
    >> from prefect import Client; c = Client()
    >> client.create_tenant(name="default")
    flavienbwk

    flavienbwk

    1 year ago
    I mean, it's when running :
    prefect server create-tenant --name default --slug default
    Chris White

    Chris White

    1 year ago
    ah i see…
    did you run all the database migrations?
    flavienbwk

    flavienbwk

    1 year ago
    Hmm, not sure, I've just run the docker-compose.yml file. Is this operation automatic ? You can directly run the compose file by git cloning this project : https://github.com/flavienbwk/prefect-docker-compose I haven't changed anything, just added a
    .env
    file containing the default values
    Chris White

    Chris White

    1 year ago
    gotcha - it’s can be made automatic if you set
    PREFECT_SERVER_DB_CMD="prefect-server database upgrade -y"
    in your environment prior to running docker-compose up; did you set that variable in your
    .env
    file?
    flavienbwk

    flavienbwk

    1 year ago
    I see ! It should be that, default is to "echo 'DATABASE MIGRATIONS SKIPPED"
    Chris White

    Chris White

    1 year ago
    ahh yes yes
    flavienbwk

    flavienbwk

    1 year ago
    Perfect, it works now! Thank you!
    Chris White

    Chris White

    1 year ago
    excellent - you’re welcome!