Thread
#prefect-community
    Mike Fransesco

    Mike Fransesco

    1 year ago
    Hello 1st message! Our company is looking at prefect and I am enquiring if there is any active support for Docker Swarm vs Kuberneties ?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Hi @Mike Fransesco, are you asking about running prefect flows (the things that do the work) on docker swarm, or running Prefect Server (the thing that orchestrates the work) on docker swarm?
    Mike Fransesco

    Mike Fransesco

    1 year ago
    oh my bad. I am looking to use the Swarm as a high availabilty/ and fail over for both the server cores and agents assigned to the tasks
    so Agent actively seeks an available server and or fails over to another in case of failure
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    So agents can be run anywhere. We provide examples/support for deploying in k8s, but if you wanted to run the docker agent in docker swarm you'd need to write your own spec (something like start a image with a
    prefect
    image and run
    prefect agent start docker
    inside it).
    For server, we provide a docker compose setup currently. I'm not familiar with docker swarm, so I'm not sure what (if anything) you'd need/want to do to make it behave the way you want to.
    We're working on improving docs/support for deploying prefect server in alternative architectures, but right now you may need to do some spelunking into the existing docker-compose file if you're trying to do something complicated that we don't currently expose in a nice way. https://github.com/PrefectHQ/prefect/blob/master/src/prefect/cli/docker-compose.yml
    Mike Fransesco

    Mike Fransesco

    1 year ago
    The end game is fail over ability. A task will execute and complete on a server someplace some how..in the case a docker server instance fails, another one will be available to take on the task. Hope that makes more sense or perhaps this is not something that is possible and I am barking up the wrong tree
    But I will read your links., thanks
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Prefect has 3 components: • Flow runs, the thing that does the work. You shouldn't have to do anything to make these robust, the agent/server architecture is designed to handle flow run failures and supports retries if needed. • Agents, the thing that starts flow runs. These need to be deployed by the user in a robust way. We provide docs and tooling for k8s and local deploys, but any backend should work fine. Sounds like you want to use docker swarm for this. • Server, the thing that manages multiple flows, and submits flow runs to the agents. We support a docker-compose setup for this. Server should be robust to restarts of any of its component services, whatever you deploy the services on should be able to handle restarting failed services.
    Mike Fransesco

    Mike Fransesco

    1 year ago
    ty
    also.. the gui interface which shows the scheduled and active tasks running and there status etc.. is there any development on providing a user friendly GUI into it that is restrictive on what can be done ? and allow for human interaction, ie entering data of forms between tasks, etc ?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    is there any development on providing a user friendly GUI into it that is restrictive on what can be done ? and allow for human interaction, ie entering data of forms between tasks, etc ?
    I'm not sure I follow the question, so a few assorted points: • Prefect Server (the free, open source thing you can install on your own hardware) has no user or auth model, all users have the same access and privileges. Prefect Cloud (our product, which also has a free tier) does have an auth model, where you can restrict access to a tenant to certain users. • The UI is shared between cloud and server (the same UI code runs on cloud as server, and lives here https://github.com/PrefectHQ/ui). • When running flow runs, you can enter parameters in the UI as a JSON field. We don't currently support a table view on this. • The UI should be extensible and can be self hosted, so if you want to add your own extensions you're free to.
    Mike Fransesco

    Mike Fransesco

    1 year ago
    ok