<@ULVA73B9P> How is the /tmp/runner_storage direct...
# ask-marvin
s
@Marvin How is the /tmp/runner_storage directory organized? How are the directory names under /tmp/runner_storage generated? If two users run the same workflow with the same parameters is it possible for both users to same file name generated in the /tmp/runner_storage directory or are they guaranteed to be unique?
m
thought for 168.2 seconds
Great question — here’s how Prefect 3.x handles
/tmp/runner_storage
on disk. Summary - Structure:
/tmp/runner_storage/<runner-uuid>/...
- Runner directory naming: a new UUID4 is generated per Runner instance - Uniqueness: different Runner processes always get different top-level directories; subfolders are reused within a single Runner for the same storage config - Cleanup: the entire
<runner-uuid>
directory is removed when the Runner shuts down normally Details How it’s organized - When a Runner starts, it creates a temp working root at: -
<system temp dir>/runner_storage/<uuid4>
- On Linux/macOS this is typically
/tmp/runner_storage/<uuid4>
- Under that UUID directory, Prefect writes content for the storage you attach to the Runner (e.g., checked-out Git repos, downloaded files, etc.). The exact layout of these subfolders depends on the storage type, but they are all rooted under the per-runner UUID directory. How names are generated - The top-level folder name is a UUID4 created when the Runner is instantiated. It is not based on the flow name, parameters, or storage — just a new random UUID. - Subdirectories inside the UUID folder are created per storage configuration. If you use the same storage configuration multiple times within the same Runner instance, it will reuse the same subfolder under that UUID root (by design). Are names unique across users/runs? - Across different Runner processes (e.g., two users, two shells, two machines), the UUID ensures the top-level directory is unique. Two users running the same workflow with the same parameters will not share the same absolute temp path. - Within a single Runner process, the storage subdirectory for a given storage configuration may be reused across multiple flow runs. If you require per-run isolation at the filesystem level, start a fresh Runner per run (or vary the storage configuration so it writes to a different subfolder). Cleanup - The temp directory is created when the Runner starts and is deleted recursively when the Runner shuts down normally. - If the process is killed (e.g., SIGKILL) or crashes before shutdown, cleanup may not occur and you may see orphaned directories under
/tmp/runner_storage
. Extra notes - The base root uses Python’s `tempfile.gettempdir()`; there isn’t a Prefect setting to configure the base path. - The Runner exports the chosen temp root to subprocesses as
PREFECT__STORAGE_BASE_PATH
for internal use. If you’re asking specifically about “same file name,” note: - File names inside the storage content (e.g., a repo’s files) will naturally be the same for the same code, but the absolute paths differ across Runner instances because of the unique
<uuid4>
segment. - Do you need per-flow-run unique directories even within a single Runner? If so, I can suggest patterns to enforce that.