Hi. Very noob question but can't get my head around it. So if we do a Docker deployment, does that mean that we need the orchestration layer running on some other server as well? Can we make that Docker deployment self-running after initiation? The use case: we have a data analytic product with several Docker containers in it and we initiate all using a Docker compose. (We install this in each individual customer premises). Need to use Prefect as our transformation layer (we currently use cron tabs) and it's good if I can make a single Docker container dedicated to Prefect so that it does periodic transformations by itself. Is this achievable with Prefect? Any tutorial will be much appreciated. Thanks p.s.: we evaluated Airflow for this but decided that it introduce too many docker containers for a simple set of tasks

Regardless of how you execute your prefect flow deployments (process, docker, etc...) my current understanding is that you still need to run a prefect agent somewhere. We're currently running the agent on a virtual machine and it's executing our flows in docker containers. I imagine you could add a prefect agent setting to your docker-compose configuration, but I don't currently know if prefect supports docker-in-docker at the moment.

Hey @Homesh Wathsalya Just adding some additional input to Adams answer 😄, In Prefect 2.0 you don't necessarily need an agent for your flows to communicate with prefect cloud that said you do lose out on the orchestration tools available through workqueues/agents (scheduling, etc.). This is just a thought around this but you may be able to setup an Agent in the Docker Container and then build a deployment with the process infrastructure block pointing to it that would then in theory execute the flow in the same docker container.

@Adam Eury @Mason Menges Thanks Guys. Appreciate the answers 🙂