Thanks for the detailed logs — they’re super helpful. What’s happening - The error is coming from Firefox failing to start because the runtime environment your flow is running in does not have GTK libraries installed: -

libgtk-3.so.0: cannot open shared object file: No such file or directory

- This is not a Prefect issue; it’s an OS-level dependency problem inside the environment where your flow is executed. Where your flow is actually running - In Prefect 3.x, deployments do not run in Prefect Cloud. Cloud only orchestrates. - Your deployment is picked up by a worker that’s attached to a work pool, and the code runs wherever that worker runs (your machine, a VM, a container, Kubernetes, etc.). - If you’re using the “default-work-pool”, it’s typically a Process-type pool — meaning it expects a worker process on infrastructure you control. If you’re using a Managed/hosted option, the base image still won’t include GUI libs by default. How to fix it (pick one path) Option A: Use a Docker work pool with a browser-ready image (recommended) - Use an image that already includes Firefox + all deps (e.g., a Playwright image): - Example:

<http://mcr.microsoft.com/playwright:v1.46.0-jammy|mcr.microsoft.com/playwright:v1.46.0-jammy>

- Point your deployment at a Docker work pool and set the image in the deployment. For example, if you’re deploying from source in Prefect 3.x:

from prefect.flows import Flow

  flow = Flow.from_source(
      source="<https://github.com/your-org/your-repo>",
      entrypoint="path/to/flow.py:your_flow",
  )

  flow.deploy(
      name="web-scraper",
      work_pool_name="docker-pool",
      image="<http://mcr.microsoft.com/playwright:v1.46.0-jammy|mcr.microsoft.com/playwright:v1.46.0-jammy>",
      build=False,   # using a public image, so no build/push needed
      push=False,
  )

- If you need Camoufox specifically, you can still base on a Playwright image and

pip install camoufox

in a custom image, or run a setup step in your container entrypoint. Option B: Keep using a Process work pool and install system packages on the worker host - Install the required libs on the machine where your worker is running (Debian/Ubuntu example):

sudo apt-get update && sudo apt-get install -y --no-install-recommends \
    libgtk-3-0 libglib2.0-0 libdbus-glib-1-2 libx11-xcb1 libnss3 \
    libxrandr2 libxcomposite1 libxdamage1 libxfixes3 libxkbcommon0 \
    libasound2 fonts-liberation libatspi2.0-0 libgbm1 libxshmfence1

- If you’re using Playwright, you can also run:

playwright install --with-deps firefox

which installs the browser and most deps on Debian-based systems. Option C: Build a custom Docker image with your exact stack - Example Dockerfile if you want

prefect + playwright + camoufox

baked in:

FROM <http://mcr.microsoft.com/playwright:v1.46.0-jammy|mcr.microsoft.com/playwright:v1.46.0-jammy>

  # Prefect + your deps
  RUN pip install --no-cache-dir prefect camoufox

  # Optional (Playwright already includes needed packages, but this is fine if you base on a slimmer image):
  # RUN apt-get update && apt-get install -y --no-install-recommends \
  #   libgtk-3-0 libglib2.0-0 libdbus-glib-1-2 libx11-xcb1 libnss3 \
  #   libxrandr2 libxcomposite1 libxdamage1 libxfixes3 libxkbcommon0 \
  #   libasound2 fonts-liberation libatspi2.0-0 libgbm1 libxshmfence1 && \
  #   rm -rf /var/lib/apt/lists/*

- Push the image to your registry and point your Docker/Kubernetes work pool at it. A couple of helpful references - Prefect 3 deployments overview and prefect.yaml: prefect.yaml guide - Infrastructure choices (Docker/Kubernetes): Kubernetes how-to (concepts apply similarly for Docker) Questions so I can tailor the exact steps - Which work pool type are you using right now (Process, Docker, Kubernetes, or Managed)? - Do you have a worker running on your own machine/VM, or are you using a Prefect-managed option?

- Are you open to switching this deployment to a Docker work pool with a browser-ready image? Once I know that, I can give you precise, copy/paste steps for your setup.

Short answer: not in the way you intend. Why -

prefect.deployments.steps.utility.run_shell_script

runs when you build/deploy (i.e., during

prefect deploy

), on the machine that is performing the deploy. It does not run on the worker at flow runtime. - So this won’t install GTK libraries into the environment where your flow actually executes unless your worker is the exact same machine that’s running

prefect deploy

(and even then, using

sudo

in a non-interactive step can fail). What to do instead - If your work pool is Docker or Kubernetes: bake these libs into the container image and set that image on the deployment/work pool. - Example base image:

<http://mcr.microsoft.com/playwright:v1.46.0-jammy|mcr.microsoft.com/playwright:v1.46.0-jammy>

already includes Firefox deps and is a good fit for Camoufox + Playwright. - Then set

image=

in

flow.deploy(...)

or via

prefect.yaml

for the deployment. - If your work pool is Process (worker runs on a VM/bare host): install the packages on the worker host itself (via your provisioning/CM tool or manual install), not in deploy steps. - Avoid trying to apt-get at runtime inside a managed/ephemeral environment — you usually won’t have

sudo

or root privileges, and it’s slow and brittle. If you want, I can give you a concrete

prefect.yaml

+ Dockerfile snippet to: - build a custom image with

prefect + camoufox

on top of a Playwright base, and - reference it from a Docker work pool deployment. Helpful references - Prefect 3 deployments and prefect.yaml: prefect.yaml guide - Infra choices and images: Kubernetes how-to (concepts apply similarly for Docker) If you can share: - your work pool type (Process, Docker, Kubernetes, or Managed), and - whether you can use a custom image, I’ll tailor exact, copy/paste steps for your setup.

Got it — if you want to avoid Docker/Kubernetes, you’ll want to run a Process worker and install the system libs on the machine where that worker runs. Key clarification - You cannot “install on a work pool”. A work pool is just a queue. The code executes on a worker process you run somewhere. - So you must install the GTK/Firefox deps on the host that is running your worker for

default-work-pool

. How to proceed 1) Make sure you’re running a worker for that pool - Start it on a machine/VM you control (Ubuntu/Debian example):

prefect worker start -p default-work-pool -n web-scraper-worker

(CLI reference: run

prefect worker start --help

) 2) Install the required system libraries on that same host - On Ubuntu/Debian:

sudo apt-get update && sudo apt-get install -y --no-install-recommends \
    libgtk-3-0 libglib2.0-0 libdbus-glib-1-2 libx11-xcb1 libnss3 \
    libxrandr2 libxcomposite1 libxdamage1 libxfixes3 libxkbcommon0 \
    libasound2 fonts-liberation libatspi2.0-0 libgbm1 libxshmfence1

- If you’re leveraging Playwright under the hood, you can also do:

pip install playwright
  playwright install --with-deps firefox

That installs both the browser and most required OS deps on Debian-based systems. 3) Restart the worker after installing packages - Just Ctrl+C and re-run the

prefect worker start ...

command. Notes - The

prefect.deployments.steps.utility.run_shell_script

approach won’t help here because that runs at deploy-time (on the machine running

prefect deploy

), not on the worker at flow runtime. - If your worker is running somewhere you can’t install system packages (e.g., an ephemeral or locked-down environment), you’ll need to run the worker on a VM you control so you can install these deps. Handy references - Work pools and workers overview: use the Prefect UI’s Work Pools page to see which workers are connected, or inspect via CLI:

prefect work-pool inspect default-work-pool

- Prefect deploy/prefect.yaml basics: prefect.yaml guide If you’re unsure which machine is currently running your worker for

default-work-pool

, let me know how you’re starting it and I’ll help you pinpoint it and get the exact install commands dialed in.

Short answer: yes. In Prefect 3.x, a “work pool” (including the default one) is a control‑plane object — it doesn’t “run” anywhere itself. The thing that runs on your machine is the worker process that polls the pool. You can run a Prefect worker on any Linux distro where you can install Python and Prefect, including Debian- and Arch-based systems. Typical flow if you’re using a process-type pool: - Install Prefect on your machine - Start a worker pointing at your pool (often named “default”) Example:

pip install -U prefect
prefect worker start -p "default"

Useful links: - Workers (what runs on your machine) - Work pools and workers overview If you know which pool type you’re targeting (process, docker, kubernetes, etc.) or whether you’re on Prefect Cloud vs OSS, I can tailor setup notes for that environment.