Karthik R
12/10/2025, 6:58 PMNate
12/10/2025, 7:03 PMMarvin
12/10/2025, 7:03 PMNate
12/10/2025, 7:04 PMprefect server start
• hosting a postgres database
• hosting a redis instance
• hosting some client machines that run prefect worker startNate
12/10/2025, 7:04 PMMarvin
12/10/2025, 7:11 PMprefecthq/prefect:3-latest
- Command:
prefect server start --host 0.0.0.0 --port 4200 --no-services
- HTTP ingress on port 4200 (public or private per your needs)
- Configure env/secret settings for Postgres/Redis per the self-hosted guide above
2) Services App
- Image: prefecthq/prefect:3-latest
- Command:
prefect server services start
- No ingress required; ensure it can reach Postgres and Redis
- Notes:
- Keep at least 1 replica for both, otherwise ACA scale-to-zero will stop the API/scheduler.
- Add liveness/readiness probes and resource requests/limits.
- Workers and flow run infrastructure
- Strongly recommended: use Prefect’s Azure Container Instances integration for execution, even if the server runs on ACA. Create a work pool and use an ACI worker.
- Create a push work pool (serverless):
prefect work-pool create --type azure-container-instance:push --provision-infra my-aci-pool
- Or use a hybrid worker (polling) running anywhere you like (VM, container app, AKS node, etc.):
prefect work-pool create --type azure-container-instance my-aci-pool
prefect worker start --pool my-aci-pool --type azure-container-instance
- If you insist on a worker inside ACA, run another ACA app with the worker process:
prefect worker start --pool <your-pool> --type azure-container-instance
Keep min replicas > 0 to avoid cold starts and missed polls.
- Networking and settings
- Make sure the worker can reach the Prefect API URL and Azure APIs for provisioning ACI jobs.
- Store secrets (DB URL, Redis, Azure creds) as ACA secrets and map to env vars (see self-hosted guide for Prefect server settings).
- Consider disabling Dapr sidecars unless you need them; they add overhead.
Tradeoffs and caveats on ACA
- Not tested/supported by Prefect; you’ll own the operational burden.
- Scale-to-zero will break the server and/or worker; keep min replicas > 0.
- Multi-container docker-compose patterns don’t translate 1:1; you’re managing multiple ACA apps and wiring them together.
- Latency: ACA cold starts can add seconds to startup. Pre-warm server and worker replicas to mitigate. For execution, ACI push pools introduce provisioning latency per run; consider concurrency and job TTL tweaks to balance cost vs. latency.
Should we add docs?
- I’d recommend:Marvin
12/10/2025, 7:11 PMBring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by