https://prefect.io logo
Join Slack
Powered by
Hey, first question in here - how do I get started...
# prefect-server
a
Hey, first question in here - how do I get started if I'd like to deploy exclusively on docker? Does it really require python on host machine to run? Would prefer a simple docker-compose.yml to start and not have to install python if possible. Thanks!
k
Hey @Alderson, I answered you in the introductions channel but we can keep the conversation going here. Yes you need Python. If you have a container that you already want to run, you can use the StartContainer task with PRefect.
z
Hey @Alderson -- if you're just trying to deploy Prefect Server without Python on the host machine you can generate the compose yaml on another machine first.
prefect server config > docker-compose.yaml
prefect server config
takes roughly the same settings as 
prefect server start
If you're just getting started though, it's almost always easier to use the free Cloud tier first then switch to Server if you decide you need your own deployment.
a
oh, that's wonderful, thanks for that! I'll play around with that and see how it goes
Could you make it an FR to have docker-compose.yml available on the repo?
z
The base docker-compose file is available in the repo, but it's filled with settings from the config and the CLI which is why we provide a tool to generate one as well
a
If you're just getting started though, it's almost always easier to use the free Cloud tier first then switch to Server if you decide you need your own deployment.
Is there any difference between the cloud and the self hosted one as it stands currently? I understand UI was recently open sourced?
z
The UI has been open source for a while now. There are a few differences, but they're mostly auth related or non-beginner feature related like if you want automated notifications or global enforcement of limits like concurrent flow runs
a
Thanks, and if I understand this correctly, coming from a bit of airflow background where tasks are run by the same machine hosting the web ui, in prefect, the ui and scheduler is one service on one host (usually the cloud), and the actual tasks would be on a separate machine, correct? If this machine is offline, then the tasks do not get executed, yes? And prefect does not run it, only manage the execution. Am I understanding this correctly?
z
Yep!
You run your "agent" on a separate machine which polls the "scheduler" for ready runs then deploys runs to the machine it is on (local/docker agents) or to other infrastructure (ecs/kubernetes agents)
If you're running Prefect Server, it could be the same machine -- it's just frequently separate.
a
The Docker agent executes flow runs in individual Docker containers.
And so, let's say we have 5 flows to run, then 
prefect agent docker start
will create 5 containers, one for each flow?
z
Yep!
It creates the containers in parallel as well -- once the agent deploys the flow it generally cedes responsibility for it
a
Btw michael, thanks a whole bunch for your replies, it makes the onboarding a whole lot less headache-inducing! 😄
cedes responsibility to the server/cloud?
so the cloud can stop the docker container when the flow is stopped?
z
No problem!
Cedes responsibility as in it does not usually require the agent to continue running for the flow run to succeed (I think a local agent is an exception since they are subprocesses) -- the agent is designed to be lightweight
If the flow is cancelled via the backend, the flow runner in the container will detect it and exit the container.
a
it does not usually require the agent to continue running for the flow run to succeed
Not too sure I understand this very well - if the agent is offline, how does the flow run?
z
The agent is required to start the flow but not for its continued operation
i.e. Agent is running -> Flow run is ready -> Agent deploys flow run as a K8s job -> Agent is killed -> Flow continues running
2 Views
Open in Slack
PreviousNext
Powered by