curious if there's any helpful design approaches people have adopted to structuring their workspace in prefect. currently setting prefect up, have a single workspace, and would like to use it across environments for cost reasons. i'm designing a naming convention/worker pool strategy for each env/pipeline, but curious if others have gone down this path and adopted something similar/different/

hi @David DeStefano yeah i think namespacing deployments/work pools is the most common way to do this within a single workspace tags might also be useful for bookkeeping

also curious what general thoughts are on managing work pools to single "pipelines" in my architecture a repo represents a pipeline with a specific responsibility. consider "integration-etl-pipeline" "ml-scoring-pipeline" "ml-training-pipeline" "app-reverse-etl-pipeline" as examples. my instinct is to have each deployment map to a top-level flow that acts as the orchestrator of child flows/tasks. i'm also considering making each pipeline have its own work pool and worker, based so the needs of the service can be isolated. the design assumes all flows/tasks will share a common deployment. the only case where a sub-flow would have its own deployment would be in my scoring pipeline. curious if the following makes sense or is overkill: one-to-one (worker pool + worker) with (pipeline + environment)

fwiw since i'm missing full context, but this makes high level sense to me! i'd just say • i like a work pool per infra type, per environment, as in 4 total if flows might run in docker or ECS in either staging or prod • with k8s/ECS/ACI for example, each deployment can use an

image

or namespace or cpu/mem requests decided at flow run creation time (e.g.

run_deployment

accepts

job_variables

like

image

that you might want in your scoring pipeline) so 1 worker can usually handle submission of all your runs in that "cluster" (whichever container orchestrator you're using) • if you don't want/need dynamic dispatch of infra for something, i like .serve in a container or just on a machine somewhere (no work pool / worker required) here's a template repo: https://github.com/zzstoatzz/prefect-pack

thanks @Nate - appreciate your input and will check the template repo out.