Short answer: yes, that pattern is common and works well—run a Prefect Server on EC2 and isolate dev/stage/prod with separate work pools, workers, and Docker images. Whether you should use a single Server or one per environment depends on your isolation needs. Recommended starting point (Prefect 3.x): - Single Prefect Server (EC2) + Postgres (ideally RDS) + S3 for results/artifacts. - Create one work pool per environment (dev, stage, prod). - Run workers per environment that only poll their pool. - Use separate Docker images/tags per environment for your flow code. - Deploy flows to the appropriate pool. When to use one Server for all environments: - You want lower ops overhead. - You don’t need hard isolation between environments. - You’re fine with all envs visible in the same UI. Pattern: - Pools: dev, stage, prod - Workers: one or more per pool - Images: repo:tag per env (e.g., etl:dev, etl:stage, etl:prod) When to run a separate Server per environment: - You need strict isolation/blast-radius reduction. - Different upgrade cadences or DB schemas. - Access isolation (Prefect Server has limited RBAC compared to Cloud). - Regulatory/segregation requirements. CLI examples (verified): - Create work pools:

prefect work-pool create dev --type docker
prefect work-pool create prod --type docker

- Start workers (include Docker extra if needed):

uv run --with 'prefect[docker]' prefect worker start -p dev
uv run --with 'prefect[docker]' prefect worker start -p prod

- Deploy a flow to an environment-specific pool/image:

from prefect import flow
from prefect.flows import Flow

Flow.from_source(
    "<https://github.com/your-org/your-repo>",
    "flows/etl.py:etl_flow",
).deploy(
    name="etl-dev",
    work_pool_name="dev",
    image="<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/etl:dev|123456789012.dkr.ecr.us-east-1.amazonaws.com/etl:dev>",
)

Additional tips: - Use separate AWS IAM roles, ECR repos or distinct tags, S3 buckets/prefixes per env. - Manage env-specific config via Prefect profiles/variables or your secret manager (e.g., AWS Secrets Manager). - Put the Server behind an ALB with TLS; keep the DB and result storage outside the EC2 instance. Docs and references: - Prefect Server concepts: Server (Prefect 3.x) - Work pools and workers overview: Work pools & workers blog - Settings and profiles: Settings & profiles If you share your isolation requirements (RBAC, network separation, upgrade cadence), I can recommend single- vs multi-Server more definitively and sketch a tailored AWS diagram.

Short answer: 8 GB can work if your workload and concurrency are small, but I’d lean toward 16 GB if you plan to run multiple containers or heavier Python libs. Since you’re running Prefect Server and all env workers on the same box in ap-southeast-1 (Singapore), here are good options: Recommended instance types - Best general-purpose (x86, predictable performance): m7i-flex.large (2 vCPU, 8 GB). If you can bump memory, m7i.xlarge (4 vCPU, 16 GB) gives nice headroom. - Cost-optimized (burstable, watch CPU credits under sustained load): t3.large or t3a.large (2 vCPU, 8 GB). Good for light/low-concurrency; avoid if you expect steady CPU. - Great price/perf if you can use Arm64 Docker images: m7g.large (2 vCPU, 8 GB) or m7g.xlarge (4 vCPU, 16 GB). Only pick this if your flow images and deps support arm64. Is 8 GB enough? - Rough budget on a single box: - Prefect Server API/UI: ~0.5–1.5 GB - Postgres (if local): ~0.5–1 GB - Worker processes: ~0.2–0.5 GB - Leaves ~5–6 GB for your flow-run containers - If your flows are small and you run 1–2 at a time, 8 GB is fine. If flows are pandas/ML-heavy or you want >2 concurrent runs, 16 GB is much safer. Storage and other tips - EBS: gp3 100–200 GB (base 3,000 IOPS) to hold Docker images, logs, and DB. Consider a separate EBS volume for Postgres data if it’s on-box. - If possible, use RDS for Postgres to free memory/IO and reduce blast radius. - Prefer non-burstable M-family for steady workloads. Burstable T-family is fine for dev/light use. - If you consider Arm (m7g), ensure your Docker images are built for arm64. If you share your expected concurrency and the heaviest library set used in your flows, I can size this more precisely. References: - Prefect Server overview: Server (Prefect 3.x) - Work pools and workers: Work pools & workers blog

✅ thought for 47.3 seconds

Great—that simplifies sizing. With Postgres on RDS and images in ECR, 8 GB RAM is much more workable for a single EC2 running Prefect Server + workers. Practical instance recommendations (ap-southeast-1): - Default pick (general-purpose, steady performance): m7i-flex.large (2 vCPU, 8 GB). If m7i-flex isn’t available in Singapore, use m7i.large. If you want more headroom or expect >2 concurrent flow runs, m7i.xlarge (4 vCPU, 16 GB) is a great upgrade. - Cost-optimized (burstable; fine for light/occasional workloads): t3a.large or t3.large (2 vCPU, 8 GB). Use T-class only if your CPU usage is not sustained—enable Unlimited and set CloudWatch alarms on CPUCreditBalance. - Best price/perf if you can build arm64 images: m7g.large (2 vCPU, 8 GB) or m7g.xlarge (4 vCPU, 16 GB). Only pick Graviton if your Docker images/deps support arm64. Quick sizing guidance - 1–2 small/medium flows at a time (ETL with moderate pandas/IO): 2 vCPU, 8 GB is fine. - Heavier libs (pandas/pyarrow/ML) or 3–4 concurrent runs: prefer 4 vCPU, 16 GB. - Rough memory budget on your box: Prefect Server ~1 GB, workers ~0.3–0.5 GB total, leaving ~6+ GB for flow containers on an 8 GB host. Aim ~2–3 GB per concurrent flow run. Storage and networking tips - EBS: gp3 100–200 GB for Docker layer cache + logs. gp3 gives predictable IOPS/throughput and is easy to scale. - Use RDS security groups to allow inbound from the EC2 SG only; keep RDS in private subnets. - For ECR pulls and faster/cheaper layer downloads, add VPC endpoints: - com.amazonaws.ap-southeast-1.ecr.api and ecr.dkr (Interface endpoints) - S3 Gateway endpoint (Docker pulls use S3 under the hood) - Attach an instance profile with minimal ECR permissions (ecr:GetAuthorizationToken, ecr:BatchGetImage, ecr:GetDownloadUrlForLayer). If you share your typical concurrent flow runs and the heaviest libraries in your images, I can give a firmer pick (and even estimate CPU/mem reservations per container). References: - Prefect Server overview: Server (Prefect 3.x) - Work pools and workers: Work pools & workers blog

Got it — that setup will work well. Key points for Prefect 3.x: - Redis is not required by Prefect Server or workers. - Using AWS RDS for Postgres is supported; use the async driver. How to configure Prefect Server to use RDS Postgres - Ensure the

asyncpg

driver is installed on your EC2 host:

pip install asyncpg

- Set Prefect database settings via environment variables (recommended for server processes). Use the async driver `postgresql+asyncpg`: Option A: Set individual fields

export PREFECT_SERVER_DATABASE_DRIVER=postgresql+asyncpg
export PREFECT_SERVER_DATABASE_HOST=<your-rds-endpoint>
export PREFECT_SERVER_DATABASE_PORT=5432
export PREFECT_SERVER_DATABASE_USER=<username>
export PREFECT_SERVER_DATABASE_PASSWORD=<password>
export PREFECT_SERVER_DATABASE_NAME=<database>

# Optional SQLAlchemy pool tuning
export PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE=10
export PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW=10
export PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_RECYCLE=3600
export PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT=30

# Enable TLS to RDS (recommended)
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_ENABLED=true
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CA_FILE=/path/to/aws-rds-ca.pem
# Optional, if you use client certs (not typical for RDS):
# export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CERT_FILE=/path/to/client-cert.pem
# export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_KEY_FILE=/path/to/client-key.pem
# export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CHECK_HOSTNAME=true

# Migrate DB schema automatically on server start (default is true)
export PREFECT_SERVER_DATABASE_MIGRATE_ON_START=true

Option B: Use a full connection URL

export PREFECT_SERVER_DATABASE_CONNECTION_URL="postgresql+asyncpg://<user>:<pass>@<host>:5432/<db>"
# For TLS with asyncpg, prefer the TLS settings above over URL params.

Initialize/upgrade the database schema - You can rely on

PREFECT_SERVER_DATABASE_MIGRATE_ON_START=true

, or run an explicit upgrade:

prefect server database upgrade

Start the server

prefect server start

Running dev/stage/prod on the same EC2 - Use separate Prefect profiles and separate RDS databases (or schemas) for isolation. For example:

prefect profile create dev && prefect profile use dev
prefect config set PREFECT_SERVER_DATABASE_NAME=prefect_dev

prefect profile create stage && prefect profile use stage
prefect config set PREFECT_SERVER_DATABASE_NAME=prefect_stage

prefect profile create prod && prefect profile use prod
prefect config set PREFECT_SERVER_DATABASE_NAME=prefect_prod

- If you run multiple servers concurrently on the same host, set unique API ports per profile and bind addresses to avoid conflicts (e.g.,

PREFECT_SERVER_API_PORT

), and ensure each points to its own RDS DB. References - Self-hosted guide: Prefect 3 self-hosted - Settings reference: Prefect 3 settings reference Questions - Do you want me to provide a minimal systemd unit or docker-compose example for the server with the RDS TLS config? - Will you run multiple server instances at the same time on the EC2 host (one per env), or a single server with logical separation?