Short answer: yes, this is a great pattern and it works with Prefect Cloud. Your web app can submit work to Prefect Cloud and a sidecar worker process can poll a work pool and execute those runs. And no—spinning up additional workers does not consume “work pool” count; multiple workers can share a single work pool. How it fits together - Your web server submits a flow run to a work pool in Prefect Cloud. - A worker (running as a sidecar on the same host/pod) polls that work pool and picks up the run. - You can run many workers against the same work pool; workers do not “use up” or create additional pools. Plan/limits note - Creating a worker does not create a new work pool. You can have multiple workers on a single pool. - Prefect Cloud plans do not document a hard limit on number of work pools or workers per pool. What’s typically rate-limited are API calls, logs/events, and data retention. See Rate limits. - If you want one worker per app instance, you can still keep a single shared pool (optionally separate queues per instance if you want stickiness). Minimal setup 1) Create or pick a work pool (for a sidecar that executes code locally, “process” is simplest). You can do this in the UI or with the CLI, then start a worker:

prefect worker start --pool my-sidecar-pool --name app-1-sidecar

Make sure the sidecar has

PREFECT_API_URL

and

PREFECT_API_KEY

set for your Cloud workspace. 2) In your app, bind your flow to that pool and submit to it:

from prefect import flow
from prefect.flows import bind_flow_to_infrastructure
from prefect.workers.process import ProcessWorker

@flow
def my_job(x: int) -> int:
    return x + 1

# one-time bind at import/startup
infra_flow = bind_flow_to_infrastructure(
    my_job, work_pool="my-sidecar-pool", worker_cls=ProcessWorker
)

# inside your request handler
fut = infra_flow.submit_to_work_pool(x=42)
result = fut.result()  # optional: wait for completion

Alternatively, you can submit directly via the API client if you prefer:

from prefect import get_client

async def submit():
    async with get_client() as client:
        fr = await client.create_flow_run(
            flow=my_job, 
            work_pool_name="my-sidecar-pool",
            parameters={"x": 42},
        )

Key docs - Workers (concepts): Workers - Work pools (concepts): Work pools - Submit flows to dynamic infrastructure (covers .submit_to_work_pool): Submit flows directly to dynamic infrastructure - API client (create_flow_run): PrefectClient.create_flow_run A few practical tips - Ensure the sidecar container has the same code/dependencies as your app’s flow code; the Process worker will run flows as subprocesses inside that container. - If you want each app instance to run only its own jobs, use a separate work queue per instance and target that queue on submission. - For Docker/Kubernetes workers, install the appropriate extras (e.g.,

prefect[docker]

,

prefect[kubernetes]

) and use a Docker/K8s work pool instead of “process”. If you share your runtime (Docker/K8s) and whether you want strict stickiness to each app instance, I can suggest an exact pool/queue layout and a sidecar manifest.

Great question. Short answer: yes, you can run a background task worker as a sidecar and have your web server submit background tasks to Prefect Cloud—but there’s one key difference from flow workers. - Background task workers do not use work pools. They connect directly to Prefect Cloud via a WebSocket subscription and listen for tasks you submit with

.delay()

. - Because they don’t use work pools, they do not count toward any “work pool” limits. You can run multiple background task worker sidecars without creating or consuming work pools. - You still need

PREFECT_API_URL

and

PREFECT_API_KEY

in the sidecar so it can connect to your Cloud workspace. Minimal example - Sidecar (worker):

from prefect import task
from prefect.task_worker import serve

@task
def send_email(to, body):
    ...

if __name__ == "__main__":
    serve(send_email, limit=4)  # concurrency per worker

- Web server (submitting work to Cloud):

send_email.delay(to="<mailto:user@example.com|user@example.com>", body="Hello!")

Key differences vs. flow workers - Background tasks: push-style via subscription; no work pools/queues; scale by running more task worker processes/pods and tuning

limit=

. - Flow runs: pull-style via work pools and workers; submit with

.submit_to_work_pool()

or deployments. Docs - How to run background tasks: Run background tasks - End-to-end web app + background tasks (sidecars/containers): Deploy a web app powered by background tasks - Cloud rate limits (apply to both approaches): Rate limits If you’re deciding between background tasks vs. flows + work pools for your sidecar pattern, I’m happy to help weigh the tradeoffs for your app (e.g., infra separation, scaling, retries, observability).