https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I face this error when worker start a...
# ask-marvin
t
@Marvin I face this error when worker start a docker image, is it because of the permission in my flow docker image itself or prefect docker image? File "/usr/local/lib/python3.11/pathlib.py", line 1116, in mkdir os.mkdir(self, mode) OSError: [Errno 30] Read-only file system: '/home/atlas/prefect'
m
✅ thought for 128.8 seconds
Short answer: it’s coming from inside your flow’s container. Prefect tries to create its home directory at startup, and your container’s filesystem at 
/home/atlas
is read-only, so the 
mkdir
fails. Details: - In Prefect 3.x the home directory defaults to 
~/.prefect
. Prefect creates this directory on startup. - In your case it’s trying to create 
/home/atlas/prefect
, which means 
PREFECT_HOME
is likely set to that path (or your image changed the default). Either way, that location isn’t writable in the running container. How to fix Pick one of these: 1) Point Prefect home to a writable path - Set an env var in your job/pool/deployment so the runtime uses a writable directory:
Copy code
PREFECT_HOME=/tmp/prefect
- For Docker/K8s/ECS jobs, add this to the worker pool’s default job variables or the deployment’s job variables/env. 2) Make the target path writable in your image - If you run as a non-root user (e.g., 
atlas
), create and chown the directory in your Dockerfile:
Copy code
ENV PREFECT_HOME=/home/atlas/.prefect
RUN mkdir -p $PREFECT_HOME && chown -R atlas:atlas $PREFECT_HOME
- Then ensure the container is not started with a read-only root filesystem (or mount a writable volume there). 3) Mount a volume at the home path - When you start the container, mount a volume to the Prefect home path:
Copy code
-v prefect-data:/home/atlas/.prefect
- Or if you must keep a read-only rootfs, add a writable tmpfs/emptyDir at that path. A couple of checks: - Verify whether you’ve set 
PREFECT_HOME
(the default would be 
~/.prefect
, not 
~/prefect
). Seeing 
/home/atlas/prefect
implies an override. - Confirm which worker type you’re using (Docker, Kubernetes, ECS) and whether it enforces a read-only root filesystem. - If you want, share your Dockerfile snippet and deployment/job variables, and I can suggest the exact change.
t
flow dockerfile
Copy code
FROM python-image AS production

ARG BUILD_USER
ARG BUILD_USER_ID
ARG BUILD_GROUP
ARG BUILD_GROUP_ID=1001
ARG BUILD_KEY_PASS
ARG UV_INDEX_ATLAS_V16_USERNAME
ARG UV_INDEX_ATLAS_V16_PASSWORD
ARG UV_INDEX_AIBOTS_USERNAME
ARG UV_INDEX_AIBOTS_PASSWORD

WORKDIR $HOME/$BUILD_USER/app

# Install dependencies
ENV UV_SYSTEM_PYTHON=1 \
    UV_PROJECT_ENVIRONMENT="/usr/local"
COPY pyproject.toml uv.lock README.md ./

USER root
RUN pip install --no-cache-dir --upgrade pip=='25.1.1' uv=='0.7.11' && \
    UV_INDEX_ATLAS_V16_USERNAME=$UV_INDEX_ATLAS_V16_USERNAME UV_INDEX_ATLAS_V16_PASSWORD=$UV_INDEX_ATLAS_V16_PASSWORD  \
    UV_INDEX_AIBOTS_USERNAME=$UV_INDEX_AIBOTS_USERNAME UV_INDEX_AIBOTS_PASSWORD=$UV_INDEX_AIBOTS_PASSWORD  \
    uv pip install --no-cache -r pyproject.toml --system

# Install the source code as a Python package
COPY src ./src
RUN uv pip install --system .

ENV PREFECT_HOME=/home/$BUILD_USER/prefect
RUN chown -R "$BUILD_GROUP:$BUILD_USER" "$PREFECT_HOME"

USER $BUILD_USER_ID
@Marvin flow dockerfile
Copy code
FROM python-image AS production

ARG BUILD_USER
ARG BUILD_USER_ID
ARG BUILD_GROUP
ARG BUILD_GROUP_ID=1001
ARG BUILD_KEY_PASS
ARG UV_INDEX_ATLAS_V16_USERNAME
ARG UV_INDEX_ATLAS_V16_PASSWORD
ARG UV_INDEX_AIBOTS_USERNAME
ARG UV_INDEX_AIBOTS_PASSWORD

WORKDIR $HOME/$BUILD_USER/app

# Install dependencies
ENV UV_SYSTEM_PYTHON=1 \
    UV_PROJECT_ENVIRONMENT="/usr/local"
COPY pyproject.toml uv.lock README.md ./

USER root
RUN pip install --no-cache-dir --upgrade pip=='25.1.1' uv=='0.7.11' && \
    UV_INDEX_ATLAS_V16_USERNAME=$UV_INDEX_ATLAS_V16_USERNAME UV_INDEX_ATLAS_V16_PASSWORD=$UV_INDEX_ATLAS_V16_PASSWORD  \
    UV_INDEX_AIBOTS_USERNAME=$UV_INDEX_AIBOTS_USERNAME UV_INDEX_AIBOTS_PASSWORD=$UV_INDEX_AIBOTS_PASSWORD  \
    uv pip install --no-cache -r pyproject.toml --system

# Install the source code as a Python package
COPY src ./src
RUN uv pip install --system .

ENV PREFECT_HOME=/home/$BUILD_USER/prefect
RUN chown -R "$BUILD_GROUP:$BUILD_USER" "$PREFECT_HOME"

USER $BUILD_USER_ID
m
✅ thought for 234.9 seconds
Thanks for the Dockerfile — a few things jump out that can cause exactly the “Read-only file system” at `/home/atlas/prefect`: What’s happening - Prefect tries to create 
PREFECT_HOME
on startup. In your image it’s set to 
/home/$BUILD_USER/prefect
(note: not the default 
~/.prefect
). - Your container at runtime is either: - Running with a read-only root filesystem, or - Not mounting a writable volume over 
/home/$BUILD_USER
, so Prefect can’t create the directory there. Also, in your Dockerfile the directory isn’t created and may not be owned by the runtime UID you switch to. Issues in the Dockerfile to fix - You never create the 
PREFECT_HOME
directory before switching users. - You chown using names (
"$BUILD_GROUP:$BUILD_USER"
) but switch to a numeric user (
USER $BUILD_USER_ID
). If those names don’t exist in the image, the chown will fail (or do nothing). It’s safer to chown with IDs or create the user/group. - 
WORKDIR $HOME/$BUILD_USER/app
is fragile (depends on what 
$HOME
is during build). Prefer explicit 
/home/${BUILD_USER}/app
after the user exists. - If your worker is starting the container with a read-only rootfs, this path must be backed by a writable mount. Recommended Dockerfile changes (minimal)
Copy code
# (after you've installed deps and your package)

# Create the runtime user/group if they don’t already exist
# If your base image already has them, you can skip this block.
RUN groupadd -g ${BUILD_GROUP_ID} ${BUILD_GROUP} || true && \
    useradd -u ${BUILD_USER_ID} -g ${BUILD_GROUP_ID} -m -d /home/${BUILD_USER} ${BUILD_USER} || true

# Set HOME and Prefect home
ENV HOME=/home/${BUILD_USER}
ENV PREFECT_HOME=${HOME}/.prefect

# Ensure Prefect home exists and is writable by the runtime UID:GID
RUN mkdir -p "${PREFECT_HOME}" && chown -R ${BUILD_USER_ID}:${BUILD_GROUP_ID} "${PREFECT_HOME}"

# Use a stable workdir
WORKDIR /home/${BUILD_USER}/app

# Drop privileges
USER ${BUILD_USER_ID}:${BUILD_GROUP_ID}
If your worker runs containers with a read-only rootfs - You must give Prefect a writable mount at 
PREFECT_HOME
. In a Prefect Docker work pool/deployment, set job variables like:
Copy code
{
  "volumes": [
    "prefect-data:/home/atlas/.prefect"  // omit :ro so it’s writable
  ]
  // If you explicitly set a read-only rootfs elsewhere, ensure this stays writable
  // "container_create_kwargs": { "read_only": true }  // only if you truly need it
}
- Alternatively, point Prefect to a writable tmpfs:
Copy code
{
  "container_create_kwargs": {
    "tmpfs": {
      "/tmp": ""
    }
  },
  "env": {
    "PREFECT_HOME": "/tmp/prefect"
  }
}
Quick workaround (no image changes) - Set a writable 
PREFECT_HOME
via deployment/job variables:
Copy code
PREFECT_HOME=/tmp/prefect
Next steps / sanity checks - Are you using a Prefect Docker worker? If yes, verify your work pool/deployment doesn’t set a read-only rootfs via 
container_create_kwargs.read_only: true
. - Ensure you mount a volume for Prefect home if you want state (DB, profiles, local storage) to persist across runs. - If you prefer 
/home/atlas/prefect
without the leading dot, that’s fine—just make sure it exists and is writable for the runtime UID. If you share your work pool’s job variables (volumes/container_create_kwargs) I can tailor the exact settings.
t
@Marvin atlas@a3b7720f3df2:~$ pwd /home/atlas atlas@a3b7720f3df2:~$ ls -ll total 8 drwxr-xr-x 1 atlas atlas 4096 Dec 10 07:23 atlas drwxr-xr-x 2 atlas atlas 4096 Dec 10 07:28 prefect it seems /home/atlas/prefect is writable
m
✅ thought for 72.6 seconds
Good clue — that directory looks writable in the container you inspected. The error, though, is EROFS (read‑only filesystem), which usually means the container that ran your flow was started with a read‑only rootfs or that path was mounted read‑only, not just a permissions issue. A few likely causes and how to confirm: - Different container than the failing run - Make sure you’re exec’ing into the exact container that failed (same container ID/name the worker started for that flow). Otherwise you may be checking a different runtime with a writable FS. - From the failing run’s container, run: - 
mount | grep ' / '
and 
mount | grep '/home/atlas'
to see if the root or that path is read‑only (look for 
(ro,...)
). - 
env | grep PREFECT_HOME
to confirm the path Prefect is using. - Read‑only root filesystem enabled by the worker - In a Prefect Docker work pool/deployment, this can be set via 
container_create_kwargs.read_only: true
or by mounting your home path as 
:ro
. - Inspect the failing container: - 
docker inspect <container-id> --format '{{json .HostConfig.ReadonlyRootfs}}'
- 
docker inspect <container-id> --format '{{json .HostConfig.Binds}}'
- Look for 
ReadonlyRootfs: true
or any bind like 
...:/home/atlas:ro
or 
...:/home/atlas/prefect:ro
. - Path exists but Prefect still tried to mkdir - Prefect only calls mkdir if 
PREFECT_HOME
didn’t appear to exist. If the rootfs was read‑only at startup, the mkdir would fail with EROFS even if you later created the directory in another run or container. That would explain why you now see it present and writable. Quick ways to unblock - Easiest test: set a writable home in your deployment/job variables and use a tmpfs for it:
Copy code
env:
  PREFECT_HOME: /tmp/prefect
container_create_kwargs:
  tmpfs:
    /tmp: ""
- Or mount a writable volume at your current home:
Copy code
volumes:
  - prefect-data:/home/atlas/prefect
- If you intentionally run with read‑only rootfs, keep it but ensure a writable mount backs 
PREFECT_HOME
as above. Notes on your Dockerfile - You did 
ENV PREFECT_HOME=/home/$BUILD_USER/prefect
(no dot). That’s fine, just ensure a writable mount there at runtime. - Consider creating the directory and chown with IDs before dropping privileges, and avoid the nested 
WORKDIR $HOME/$BUILD_USER/app
pattern which created 
/home/atlas/atlas/app
. If you can share: - The work pool/deployment job variables (especially 
volumes
and 
container_create_kwargs
) - A 
docker inspect
of the failing container’s 
ReadonlyRootfs
and 
Binds
…I can point to the exact setting that’s making the FS read‑only.
t
@Marvin this is the base job template
Copy code
baseJobTemplate.json: "{\n  \"variables\": {\n    \"type\": \"object\",\n    \"properties\": {\n      \"env\": {\n        \"type\": \"object\",\n        \"title\": \"Environment Variables\",\n        \"description\": \"Environment variables to set when starting a flow run.\",\n        \"additionalProperties\": {\n          \"anyOf\": [\n            { \"type\": \"string\" },\n            { \"type\": \"null\" }\n          ]\n        }\n      },\n      \"name\": {\n        \"anyOf\": [\n          { \"type\": \"string\" },\n          { \"type\": \"null\" }\n        ],\n        \"title\": \"Name\",\n        \"description\": \"Name given to infrastructure created by a worker.\"\n      },\n      \"image\": {\n        \"anyOf\": [\n          { \"type\": \"string\" },\n          { \"type\": \"null\" }\n        ],\n        \"title\": \"Image\",\n        \"default\": \"<http://registry.sgts.gitlab-dedicated.com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect:3.6.5\|registry.sgts.gitlab-dedicated.com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect:3.6.5\>",\n        \"examples\": [\"<http://docker.io/prefecthq/prefect:3-latest\|docker.io/prefecthq/prefect:3-latest\>"],\n        \"description\": \"The image reference of a container image to use for created jobs. If not set, the latest Prefect image will be used.\"\n      },\n      \"labels\": {\n        \"type\": \"object\",\n        \"title\": \"Labels\",\n        \"description\": \"Labels applied to infrastructure created by a worker.\",\n        \"additionalProperties\": { \"type\": \"string\" }\n      },\n      \"command\": {\n        \"anyOf\": [\n          { \"type\": \"string\" },\n          { \"type\": \"null\" }\n        ],\n        \"title\": \"Command\",\n        \"description\": \"The command to use when starting a flow run. In most cases, this should be left blank and the command will be automatically generated by the worker.\"\n      },\n      \"namespace\": {\n        \"type\": \"string\",\n        \"title\": \"Namespace\",\n        \"default\": \"default\",\n        \"description\": \"The Kubernetes namespace to create jobs within.\"\n      },\n      \"backoff_limit\": {\n        \"type\": \"integer\",\n        \"title\": \"Backoff Limit\",\n        \"default\": 0,\n        \"minimum\": 0,\n        \"description\": \"The number of times Kubernetes will retry a job after pod eviction. If set to 0, Prefect will reschedule the flow run when the pod is evicted.\"\n      },\n      \"stream_output\": {\n        \"type\": \"boolean\",\n        \"title\": \"Stream Output\",\n        \"default\": true,\n        \"description\": \"If set, output will be streamed from the job to local standard output.\"\n      },\n      \"cluster_config\": {\n        \"anyOf\": [\n          { \"$ref\": \"#/definitions/KubernetesClusterConfig\" },\n          { \"type\": \"null\" }\n        ],\n        \"description\": \"The Kubernetes cluster config to use for job creation.\"\n      },\n      \"finished_job_ttl\": {\n        \"type\": \"integer\",\n        \"default\": 60,\n        \"title\": \"Finished Job TTL\",\n        \"description\": \"The number of seconds to retain jobs after completion. If set, finished jobs will be cleaned up by Kubernetes after the given delay. If not set, jobs will be retained indefinitely.\"\n      },\n      \"image_pull_policy\": {\n        \"enum\": [\"IfNotPresent\", \"Always\", \"Never\"],\n        \"type\": \"string\",\n        \"title\": \"Image Pull Policy\",\n        \"default\": \"IfNotPresent\",\n        \"description\": \"The Kubernetes image pull policy to use for job containers.\"\n      },\n      \"service_account_name\": {\n        \"anyOf\": [\n          { \"type\": \"string\" },\n          { \"type\": \"null\" }\n        ],\n        \"title\": \"Service Account Name\",\n        \"description\": \"The Kubernetes service account to use for job creation.\"\n      },\n      \"job_watch_timeout_seconds\": {\n        \"anyOf\": [\n          { \"type\": \"integer\" },\n          { \"type\": \"null\" }\n        ],\n        \"title\": \"Job Watch Timeout Seconds\",\n        \"description\": \"Number of seconds to wait for each event emitted by a job before timing out. If not set, the worker will wait for each event indefinitely.\"\n      },\n      \"pod_watch_timeout_seconds\": {\n        \"type\": \"integer\",\n        \"title\": \"Pod Watch Timeout Seconds\",\n        \"default\": 60,\n        \"description\": \"Number of seconds to watch for pod creation before timing out.\"\n      }\n    },\n    \"definitions\": {\n      \"KubernetesClusterConfig\": {\n        \"type\": \"object\",\n        \"title\": \"KubernetesClusterConfig\",\n        \"required\": [\"config\", \"context_name\"],\n        \"properties\": {\n          \"config\": {\n            \"type\": \"object\",\n            \"title\": \"Config\",\n            \"description\": \"The entire contents of a kubectl config file.\",\n            \"additionalProperties\": true\n          },\n          \"context_name\": {\n            \"type\": \"string\",\n            \"title\": \"Context Name\",\n            \"description\": \"The name of the kubectl context to use.\"\n          }\n        }\n      }\n    }\n  },\n  \"job_configuration\": {\n    \"env\": \"{{ env }}\",\n    \"name\": \"{{ name }}\",\n    \"labels\": \"{{ labels }}\",\n    \"command\": \"{{ command }}\",\n    \"namespace\": \"{{ namespace }}\",\n    \"job_manifest\": {\n      \"apiVersion\": \"batch/v1\",\n      \"kind\": \"Job\",\n      \"metadata\": {\n        \"labels\": \"{{ labels }}\",\n        \"namespace\": \"{{ namespace }}\",\n        \"generateName\": \"{{ name }}-\"\n      },\n      \"spec\": {\n        \"template\": {\n          \"spec\": {\n            \"parallelism\": 1,\n            \"completions\": 1,\n            \"restartPolicy\": \"Never\",\n            \"serviceAccountName\": \"{{ service_account_name }}\",\n            \"imagePullSecrets\": [\n              { \"name\": \"gitlab-registry-secrets\" }\n            ],\n            \"securityContext\": {\n              \"runAsUser\": 1001,\n              \"runAsNonRoot\": true,\n              \"fsGroup\": 1001,\n              \"seccompProfile\": { \"type\": \"RuntimeDefault\" }\n            },\n            \"volumes\": [\n              { \n                \"name\": \"temp-volume\", \n                \"emptyDir\": {\n                  \"sizeLimit\": \"5Gi\"\n                }\n              }\n            ],\n            \"containers\": [\n              {\n                \"name\": \"prefect-job\",\n                \"image\": \"{{ image }}\",\n                \"imagePullPolicy\": \"{{ image_pull_policy }}\",\n                \"args\": \"{{ command }}\",\n                \"env\": \"{{ env }}\",\n                \"volumeMounts\": [\n                  { \"name\": \"temp-volume\", \"mountPath\": \"/tmp\" }\n                ],\n                \"securityContext\": {\n                  \"runAsUser\": 1001,\n                  \"runAsNonRoot\": true,\n                  \"readOnlyRootFilesystem\": true,\n                  \"allowPrivilegeEscalation\": false,\n                  \"capabilities\": { \"drop\": [\"ALL\"] }\n                }\n              }\n            ]\n          }\n        },\n        \"backoffLimit\": \"{{ backoff_limit }}\",\n        \"ttlSecondsAfterFinished\": \"{{ finished_job_ttl }}\"\n      }\n    },\n    \"stream_output\": \"{{ stream_output }}\",\n    \"cluster_config\": \"{{ cluster_config }}\",\n    \"job_watch_timeout_seconds\": \"{{ job_watch_timeout_seconds }}\",\n    \"pod_watch_timeout_seconds\": \"{{ pod_watch_timeout_seconds }}\"\n  }\n}\n"
m
Your message was too long, here's your message at the allowed limit: this is the base job template ```baseJobTemplate json {\n \ variables\ {\n \ type\ \ object\ \n \ properties\ {\n \ env\ {\n \ type\ \ object\ \n \ title\ \ Environment Variables\ \n \ description\ \ Environment variables to set when starting a flow run \ \n \ additionalProperties\ {\n \ anyOf\ [\n { \ type\ \ string\ } \n { \ type\ \ null\ }\n ]\n }\n } \n \ name\ {\n \ anyOf\ [\n { \ type\ \ string\ } \n { \ type\ \ null\ }\n ] \n \ title\ \ Name\ \n \ description\ \ Name given to infrastructure created by a worker \ \n } \n \ image\ {\n \ anyOf\ [\n { \ type\ \ string\ } \n { \ type\ \ null\ }\n ] \n \ title\ \ Image\ \n \ default\ \ <http //registry sgts gitlab-dedicated com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect 3 6 5\|registry sgts gitlab-dedicated com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect 3 6 5\> \n \ examples\ [\ <http //docker io/prefecthq/prefect 3-latest\|docker io/prefecthq/prefect 3-latest\> ] \n \ description\ \ The image reference of a container image to use for created jobs If not set the latest Prefect image will be used \ \n } \n \ labels\ {\n \ type\ \ object\ \n \ title\ \ Labels\ \n \ description\ \ Labels applied to infrastructure created by a worker \ \n \ additionalProperties\ { \ type\ \ string\ }\n } \n \ command\ {\n \ anyOf\ [\n { \ type\ \ string\ } \n { \ type\ \ null\ }\n ] \n \ title\ \ Command\ \n \ description\ \ The command to use when starting a flow run In most cases this should be left blank and the command will be automatically generated by the worker \ \n } \n \ namespace\ {\n \ type\ \ string\ \n \ title\ \ Namespace\ \n \ default\ \ default\ \n \ description\ \ The Kubernetes namespace to create jobs within \ \n } \n \ backoff_limit\ {\n \ type\ \ integer\ \n \ title\ \ Backoff Limit\ \n \ default\ 0 \n \ minimum\ 0 \n \ description\ \ The number of times Kubernetes will retry a job after pod eviction If set to 0 Prefect will reschedule the flow run when the pod is evicted \ \n } \n \ stream_output\ {\n \ type\ \ boolean\ \n \ title\ \ Stream Output\ \n \ default\ true \n \ description\ \ If set output will be streamed from the job to local standard output \ \n } \n \ cluster_config\ {\n \ anyOf\ [\n { \ $ref\ \ #/definitions/KubernetesClusterConfig\ } \n { \ type\ \ null\ }\n ] \n \ description\ \ The Kubernetes cluster config to use for job creation \ \n } \n \ finished_job_ttl\ {\n \ type\ \ integer\ \n \ default\ 60 \n \ title\ \ Finished Job TTL\ \n \ description\ \ The number of seconds to retain jobs after completion If set finished jobs will be cleaned up by Kubernetes after the given delay
t
@Marvin base job template
Copy code
{
  "variables": {
    "type": "object",
    "properties": {
      "env": {
        "type": "object",
        "title": "Environment Variables",
        "description": "Environment variables to set when starting a flow run.",
        "additionalProperties": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ]
        }
      },
      "name": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "null"
          }
        ],
        "title": "Name",
        "description": "Name given to infrastructure created by a worker."
      },
      "image": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "null"
          }
        ],
        "title": "Image",
        "default": "<http://registry.sgts.gitlab-dedicated.com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect:3.6.5|registry.sgts.gitlab-dedicated.com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect:3.6.5>",
        "examples": [
          "<http://docker.io/prefecthq/prefect:3-latest|docker.io/prefecthq/prefect:3-latest>"
        ],
        "description": "The image reference of a container image to use for created jobs. If not set, the latest Prefect image will be used."
      },
      "labels": {
        "type": "object",
        "title": "Labels",
        "description": "Labels applied to infrastructure created by a worker.",
        "additionalProperties": {
          "type": "string"
        }
      },
      "command": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "null"
          }
        ],
        "title": "Command",
        "description": "The command to use when starting a flow run. In most cases, this should be left blank and the command will be automatically generated by the worker."
      },
      "namespace": {
        "type": "string",
        "title": "Namespace",
        "default": "default",
        "description": "The Kubernetes namespace to create jobs within."
      },
      "backoff_limit": {
        "type": "integer",
        "title": "Backoff Limit",
        "default": 0,
        "minimum": 0,
        "description": "The number of times Kubernetes will retry a job after pod eviction. If set to 0, Prefect will reschedule the flow run when the pod is evicted."
      },
      "stream_output": {
        "type": "boolean",
        "title": "Stream Output",
        "default": true,
        "description": "If set, output will be streamed from the job to local standard output."
      },
      "cluster_config": {
        "anyOf": [
          {
            "$ref": "#/definitions/KubernetesClusterConfig"
          },
          {
            "type": "null"
          }
        ],
        "description": "The Kubernetes cluster config to use for job creation."
      },
      "finished_job_ttl": {
        "type": "integer",
        "default": 60,
        "title": "Finished Job TTL",
        "description": "The number of seconds to retain jobs after completion. If set, finished jobs will be cleaned up by Kubernetes after the given delay. If not set, jobs will be retained indefinitely."
      },
      "image_pull_policy": {
        "enum": [
          "IfNotPresent",
          "Always",
          "Never"
        ],
        "type": "string",
        "title": "Image Pull Policy",
        "default": "IfNotPresent",
        "description": "The Kubernetes image pull policy to use for job containers."
      },
      "service_account_name": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "null"
          }
        ],
        "title": "Service Account Name",
        "description": "The Kubernetes service account to use for job creation."
      },
      "job_watch_timeout_seconds": {
        "anyOf": [
          {
            "type": "integer"
          },
          {
            "type": "null"
          }
        ],
        "title": "Job Watch Timeout Seconds",
        "description": "Number of seconds to wait for each event emitted by a job before timing out. If not set, the worker will wait for each event indefinitely."
      },
      "pod_watch_timeout_seconds": {
        "type": "integer",
        "title": "Pod Watch Timeout Seconds",
        "default": 60,
        "description": "Number of seconds to watch for pod creation before timing out."
      }
    },
    "definitions": {
      "KubernetesClusterConfig": {
        "type": "object",
        "title": "KubernetesClusterConfig",
        "required": [
          "config",
          "context_name"
        ],
        "properties": {
          "config": {
            "type": "object",
            "title": "Config",
            "description": "The entire contents of a kubectl config file.",
            "additionalProperties": true
          },
          "context_name": {
            "type": "string",
            "title": "Context Name",
            "description": "The name of the kubectl context to use."
          }
        }
      }
    }
  },
  "job_configuration": {
    "env": "{{ env }}",
    "name": "{{ name }}",
    "labels": "{{ labels }}",
    "command": "{{ command }}",
    "namespace": "{{ namespace }}",
    "job_manifest": {
      "apiVersion": "batch/v1",
      "kind": "Job",
      "metadata": {
        "labels": "{{ labels }}",
        "namespace": "{{ namespace }}",
        "generateName": "{{ name }}-"
      },
      "spec": {
        "template": {
          "spec": {
            "parallelism": 1,
            "completions": 1,
            "restartPolicy": "Never",
            "serviceAccountName": "{{ service_account_name }}",
            "imagePullSecrets": [
              {
                "name": "gitlab-registry-secrets"
              }
            ],
            "securityContext": {
              "runAsUser": 1001,
              "runAsNonRoot": true,
              "fsGroup": 1001,
              "seccompProfile": {
                "type": "RuntimeDefault"
              }
            },
            "volumes": [
              {
                "name": "temp-volume",
                "emptyDir": {
                  "sizeLimit": "5Gi"
                }
              }
            ],
            "containers": [
              {
                "name": "prefect-job",
                "image": "{{ image }}",
                "imagePullPolicy": "{{ image_pull_policy }}",
                "args": "{{ command }}",
                "env": "{{ env }}",
                "volumeMounts": [
                  {
                    "name": "temp-volume",
                    "mountPath": "/tmp"
                  }
                ],
                "securityContext": {
                  "runAsUser": 1001,
                  "runAsNonRoot": true,
                  "readOnlyRootFilesystem": true,
                  "allowPrivilegeEscalation": false,
                  "capabilities": {
                    "drop": [
                      "ALL"
                    ]
                  }
                }
              }
            ]
          }
        },
        "backoffLimit": "{{ backoff_limit }}",
        "ttlSecondsAfterFinished": "{{ finished_job_ttl }}"
      }
    },
    "stream_output": "{{ stream_output }}",
    "cluster_config": "{{ cluster_config }}",
    "job_watch_timeout_seconds": "{{ job_watch_timeout_seconds }}",
    "pod_watch_timeout_seconds": "{{ pod_watch_timeout_seconds }}"
  }
}
m
Your message was too long, here's your message at the allowed limit: base job template ```{ variables { type object properties { env { type object title Environment Variables description Environment variables to set when starting a flow run additionalProperties { anyOf [{ type string } { type null }]}} name { anyOf [{ type string } { type null }] title Name description Name given to infrastructure created by a worker } image { anyOf [{ type string } { type null }] title Image default <http //registry sgts gitlab-dedicated com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect 3 6 5|registry sgts gitlab-dedicated com/wog/gvt/aibots/aibots/aibots-apps/orchestrator/prefect 3 6 5> examples [ <http //docker io/prefecthq/prefect 3-latest|docker io/prefecthq/prefect 3-latest> ] description The image reference of a container image to use for created jobs If not set the latest Prefect image will be used } labels { type object title Labels description Labels applied to infrastructure created by a worker additionalProperties { type string }} command { anyOf [{ type string } { type null }] title Command description The command to use when starting a flow run In most cases this should be left blank and the command will be automatically generated by the worker } namespace { type string title Namespace default default description The Kubernetes namespace to create jobs within } backoff_limit { type integer title Backoff Limit default 0 minimum 0 description The number of times Kubernetes will retry a job after pod eviction If set to 0 Prefect will reschedule the flow run when the pod is evicted } stream_output { type boolean title Stream Output default true description If set output will be streamed from the job to local standard output } cluster_config { anyOf [{ $ref #/definitions/KubernetesClusterConfig } { type null }] description The Kubernetes cluster config to use for job creation } finished_job_ttl { type integer default 60 title Finished Job TTL description The number of seconds to retain jobs after completion If set finished jobs will be cleaned up by Kubernetes after the given delay If not set jobs will be retained indefinitely } image_pull_policy { enum [ IfNotPresent Always Never ] type string title Image Pull Policy default IfNotPresent description The Kubernetes image pull policy to use for job containers } service_account_name { anyOf [{ type string } { type null }] title Service Account Name description The Kubernetes service account to use for job creation } job_watch_timeout_seconds { anyOf [{ type integer } { type null }] title Job Watch Timeout Seconds description Number of seconds to wait for each event emitted by a job before timing out If not set the worker will wait for each event indefinitely } pod_watch_timeout_seconds { type integer title Pod Watch Timeout Seconds default 60 description Number of seconds to watch for pod creation before timing out }} definitions { KubernetesClusterConfig { type object title KubernetesClusterConfig required [ config context_name ] properties { config { type object title Config description The entire contents of a kubectl config file additionalProperties true} context_name { type string title Context Name description The name of the kubectl context to use
6 Views
Open in Slack
PreviousNext
Powered by