oleander-whirlwind
10/09/2025, 6:33 AMMarvin
10/09/2025, 6:33 AMMarvin
10/09/2025, 6:33 AMoleander-whirlwind
10/09/2025, 6:35 AMMarvin
10/09/2025, 6:35 AMMarvin
10/09/2025, 6:41 AMls -ld /tmp
# should be: drwxrwxrwt (1777)
- If not, fix it (as root):
chmod 1777 /tmp
- Ensure /tmp is not mounted read-only or with restrictive options:
mount | grep ' /tmp '
- In a devcontainer, you can force a writable tmpfs:
- devcontainer.json (new syntax)
"mounts": ["type=tmpfs,target=/tmp"]
- or devcontainer.json (older runArgs style)
"runArgs": ["--tmpfs", "/tmp:exec,mode=1777"]
Option B — Point Prefect’s temp storage somewhere else
Prefect uses Python’s tempfile.gettempdir(), which respects the TMPDIR environment variable. Set TMPDIR to a writable path that lives on a mounted volume (e.g., your workspace).
- Create a writable directory and set TMPDIR:
mkdir -p /workspaces/your-repo/.tmp && chmod 777 /workspaces/your-repo/.tmp
export TMPDIR=/workspaces/your-repo/.tmp
- To make it persistent in a devcontainer, add to devcontainer.json:
"containerEnv": {
"TMPDIR": "/workspaces/your-repo/.tmp"
},
"postCreateCommand": "mkdir -p /workspaces/your-repo/.tmp && chmod 777 /workspaces/your-repo/.tmp"
Option C — Avoid temp storage by using a local path source
If you can clone your repo into the workspace and serve from the local path, Prefect won’t need to check out into /tmp:
from prefect import flow
flow = flow.from_source(
source="/workspaces/your-repo",
entrypoint="path/to/flow.py:my_flow",
)
flow.serve(name="my-deployment")
Quick checks to confirm the root cause
- What temp dir is Prefect using?
python -c "import tempfile; print(tempfile.gettempdir())"
- Can you write to the default runner dir?
mkdir -p /tmp/runner_storage && touch /tmp/runner_storage/test && echo OK
Reference
- Prefect stores flow code pulled from source in a temp directory; for background on the runner’s temp usage see this related discussion that shows the runner path pattern under tempfile.gettempdir() → runner_storage: flow.from_source Git pull/runner_storage context
- How to store/serve flow code from source: Store flow code for deployments
If you can share:
- Your devcontainer.json (or the relevant mounts/runArgs)
- Output of ls -ld /tmp
- Prefect version (prefect version)
I can suggest the exact minimal change.oleander-whirlwind
10/09/2025, 6:46 AMMarvin
10/09/2025, 6:46 AMMarvin
10/09/2025, 6:47 AMcurl -sf <http://host.docker.internal:4200/api/health> || echo "unreachable"
- If unreachable, add this to your devcontainer.json and rebuild:
"runArgs": ["--add-host=host.docker.internal:host-gateway"]
or point to the host’s IP directly (e.g., http://172.17.0.1:4200/api, depending on your setup).
3) Git checkout via the GitLab block failing
- You’ve set GIT_TRACE/GIT_CURL_VERBOSE — good. The runner pulls code into ${TMPDIR:-/tmp}/runner_storage/<uuid>. If the clone/pull fails, the runner will shut down quickly and you’ll see secondary websocket/observer errors.
- Please run once with DEBUG logs and share the first error above the websocket lines; look for messages from prefect.runner.storage or any subprocess errors involving git.
Quick experiments to isolate
- Force a known-writable temp dir early (before importing Prefect), just to rule out any odd temp resolution:
import os
os.environ["TMPDIR"] = "/workspaces/your-repo/.tmp"
os.makedirs(os.environ["TMPDIR"], exist_ok=True)
os.chmod(os.environ["TMPDIR"], 0o777)
- Confirm API health from inside the container:
curl -sf "$PREFECT_API_URL/health" || echo "unreachable"
- Try a simple from_source with a public repo to confirm the flow runner can clone at all:
from prefect import flow
flow = flow.from_source(
source="<https://github.com/PrefectHQ/prefect.git>",
entrypoint="examples/flows/hello_world.py:hello_world_flow",
)
flow.serve(name="smoke-test")
If this works but your GitLab block fails, we’ll focus on credentials/clone settings.
A couple more notes for your script
- You do not need to modify sys.path when using flow.from_source with a repository block; the runner loads from the cloned repo path. It shouldn’t break anything, but it’s unnecessary.
- Make sure you set all env vars before importing Prefect (you did — good).
- If you want to be explicit about the temp base, TMPDIR is the right lever; Prefect honors Python’s tempfile.gettempdir() which respects TMPDIR.
Useful references
- Store flow code for deployments and serving: https://docs-3.prefect.io/v3/how-to-guides/deployments/store-flow-code
- Context on runner temp storage pattern (uses tempfile.gettempdir()/runner_storage): https://github.com/PrefectHQ/prefect/issues/11546
Could you share:
- Prefect version (prefect version) and Python version (python --version)
- Result of curl -sf "$PREFECT_API_URL/health"
- The first error lines in DEBUG logs prior to the websocket close (especially anything about git clone/pull)
If you can switch the devcontainer to Python 3.12 and add the host-gateway mapping for host.docker.internal, that will likely resolve this quickly.oleander-whirlwind
10/09/2025, 6:50 AMMarvin
10/09/2025, 6:50 AMMarvin
10/09/2025, 6:50 AMoleander-whirlwind
10/09/2025, 6:52 AMMarvin
10/09/2025, 6:52 AMMarvin
10/09/2025, 6:53 AMrm -rf /tmp/runner_storage/*
# or fix ownership if needed
# sudo chown -R "$(id -u):$(id -g)" /tmp/runner_storage
- Or set TMPDIR to a workspace path you control (ensures fresh, consistent perms)
export TMPDIR=/workspaces/your-repo/.tmp
mkdir -p "$TMPDIR" && chmod 777 "$TMPDIR"
Do this before importing Prefect.
Option: Avoid copying .git entirely
If your GitLabRepository block supports selecting directories, set it to only copy your flow code (skipping .git):
gitlab_repository_block.directories = ["workflows"]
This typically prevents copying .git, which is where your permission error occurs.
“Rewrite the flow to check that it is pulled and not served”
If your goal is to pre-pull the code to a path you control and then serve from that local path, here’s a pattern that avoids Prefect’s temp copy:
- Step 1: Clone the repo yourself into a known-writable location (with your own credentials setup)
- Step 2: Serve from that local path
Example:
import os
import subprocess
from pathlib import Path
from prefect import flow
# Set a safe temp base first
os.environ["TMPDIR"] = "/workspaces/uvd-workflow-orchestrator-example-prefect/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
# 1) Pre-pull into a path you control
repo_dir = Path("/workspaces/uvd-workflow-orchestrator-example-prefect/_src")
if repo_dir.exists():
# start fresh to avoid stale perms
subprocess.run(["rm", "-rf", str(repo_dir)], check=True)
# Use your preferred git auth here (token/ssh). Example with HTTPS:
subprocess.run([
"git", "clone", "--depth", "1",
"<https://gitlab.example.com/group/project.git>",
str(repo_dir)
], check=True)
# 2) Serve from the local path
my_flow = flow.from_source(
source=str(repo_dir),
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
my_flow.serve(name="private-gitlab-deploy")
If you’d like to keep using your GitLabRepository block but ensure fresh perms, add this preflight at the very top (before importing Prefect):
import os
from pathlib import Path
os.environ["TMPDIR"] = "/workspaces/uvd-workflow-orchestrator-example-prefect/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
# Clean any stale runner storage
rs_base = Path(os.environ["TMPDIR"]) / "runner_storage"
if rs_base.exists():
# remove stale dirs that may be owned by another user
import shutil
shutil.rmtree(rs_base, ignore_errors=True)
Why your quick test succeeded but Prefect still fails
- Touching /tmp/runner_storage/test only proves the top-level directory is writable. Prefect is writing deeper into /tmp/runner_storage/<uuid>/.../.git/objects/pack, which may have been created by a different user/umask in a prior run. That nested directory can block overwrites even when the parent is writable.
If you’d like, share the ownership of the failing path:
ls -ld /tmp/runner_storage \
/tmp/runner_storage/38726217-2925-408a-95f6-e65d81524334 \
/tmp/runner_storage/38726217-2925-408a-95f6-e65d81524334/gitlab-repository-prefectprototypegit/.git/objects/pack
and I’ll suggest the minimal change. Also confirm if setting TMPDIR to a workspace path and cleaning runner_storage resolves it for you.oleander-whirlwind
10/09/2025, 6:57 AMMarvin
10/09/2025, 6:57 AMMarvin
10/09/2025, 6:59 AMimport os
import asyncio
from pathlib import Path
# Make sure this is set before importing Prefect to control temp base
os.environ["TMPDIR"] = "/workspaces/your-repo/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
from prefect.runner.storage import BlockStorageAdapter
from prefect_gitlab import GitLabRepository, GitLabCredentials
async def main():
repo = GitLabRepository.load("prefectprototypegit")
creds = GitLabCredentials.load("mycredential")
repo.credentials = creds
storage = BlockStorageAdapter(block=repo, pull_interval=60)
# Preflight: perform the checkout
await storage.pull_code()
# Inspect where it landed
print("Pulled to:", storage.destination)
# Optional: list files to verify
import subprocess
subprocess.run(["ls", "-la", str(storage.destination)], check=False)
asyncio.run(main())
If this succeeds, your checkout path/permissions are good. You can then pass storage into flow.from_source to reuse it:
from prefect import flow as flow_cls
flow = flow_cls.from_source(
source=storage,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
flow.serve(name="private-gitlab-deploy")
B) Use a plain Git URL to isolate the problem
- This helps check if the issue is in the GitLab block vs. general runner storage.
import os, asyncio, subprocess
from pathlib import Path
os.environ["TMPDIR"] = "/workspaces/your-repo/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
from prefect.runner.storage import create_storage_from_source
async def main():
storage = create_storage_from_source("<https://github.com/PrefectHQ/prefect.git>")
await storage.pull_code()
print("Pulled to:", storage.destination)
subprocess.run(["ls", "-la", str(storage.destination)], check=False)
asyncio.run(main())
Why this helps
- Your error shows shutil copying from a staging tmp dir into /tmp/runner_storage/<uuid>/.../.git/objects/pack and failing with EACCES. Calling pull_code directly will reproduce that permission issue in isolation, without the serve loop masking it with subsequent websocket errors.
If pull_code fails with Permission denied:
- Clean stale runner storage (created by another user/umask) and try again:
rm -rf ${TMPDIR:-/tmp}/runner_storage/*
- Or change TMPDIR to a workspace you own (as in the examples above), then rerun the preflight.
Links
- Storing/serving flow code: https://docs-3.prefect.io/v3/how-to-guides/deployments/store-flow-code
If you share the output of the preflight (especially storage.destination and ls -ld of that path), I can suggest the minimal fix.oleander-whirlwind
10/09/2025, 7:57 AMMarvin
10/09/2025, 7:58 AMMarvin
10/09/2025, 7:59 AMserve triggers. You can test just the checkout step (without serving) by calling the storage’s pull_code() — and your results confirm a permission issue on the second pull.
What’s going on
- First pull creates the destination and copies the repo.
- Second pull tries to copy over an existing .git/objects/pack/* file tree. Those pack files are often read‑only; combined with an existing tree and/or restrictive directory perms, Python’s copy will hit Errno 13.
How to verify just the checkout (no serve)
- This is the minimal preflight:
from prefect.runner.storage import BlockStorageAdapter
from prefect_gitlab import GitLabRepository, GitLabCredentials
import asyncio
async def main():
repo = GitLabRepository.load("prefectprototypegit")
repo.credentials = GitLabCredentials.load("mycredential")
storage = BlockStorageAdapter(block=repo)
await storage.pull_code()
print("Pulled to:", storage.destination)
asyncio.run(main())
If you run that twice and the second run fails with Permission denied in .git/objects/pack, it’s definitely a destination reuse/permissions problem.
Ways to fix or avoid the error
- Fresh destination each time (recommended)
- Let Prefect choose a new destination under ${TMPDIR:-/tmp}/runner_storage/<uuid>/... so you never overwrite an existing tree.
- If you previously pointed the storage to a fixed path (e.g., inside your repo), stop overriding the destination and let it be the default.
- If you want the runner storage on your mounted workspace, set TMPDIR before importing Prefect so the fresh UUID paths land there:
export TMPDIR=/workspaces/your-repo/.tmp
mkdir -p "$TMPDIR" && chmod 777 "$TMPDIR"
- Pre-clean the destination before re-pulling
- If you’re intentionally reusing a fixed location:
import shutil, pathlib
if pathlib.Path(storage.destination).exists():
shutil.rmtree(storage.destination, ignore_errors=True)
await storage.pull_code()
- Don’t copy .git at all (if your block supports directories)
- Limit to only the code dirs you need so .git isn’t copied into the destination:
repo.directories = ["workflows"] # adjust to your code dir(s)
- Fix permissions on the existing tree
- If you must reuse the same path:
chmod -R u+rwX,g+rwX /uvd-workflow-orchestrator-example-prefect/gitlab-repository-prefectprototypegit
find /uvd-workflow-orchestrator-example-prefect/gitlab-repository-prefectprototypegit/.git/objects/pack -type f -exec chmod u+w,g+w {} +
- Or just remove the existing directory before re-run.
Alternative flow (no copy step)
- If you prefer to control the checkout yourself and avoid Prefect’s copy entirely:
1) Clone to a clean path you own (fresh each time or pre-clean).
2) Serve from that path.
from prefect import flow as flow_cls
local_path = "/workspaces/your-repo/_src" # your own clone target
# ...run `git clone` here, or ensure it's clean...
f = flow_cls.from_source(
source=local_path,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
Why your /tmp/runner_storage listing looks fine but still fails
- The top-level is writable, but your failure path is inside a reused destination that already contains .git/objects/pack/* files. Those files are commonly read-only, so a second copy over the same location will fail unless you delete them or ensure the directory strategy creates a fresh destination.
References
- Storing/serving flow code: Store flow code for deployments
- Background on runner storage under tempfile/runner_storage: Related issue and notes
If you can share:
- The exact print(storage.destination) for both runsMarvin
10/09/2025, 7:59 AMls -ld for that destination and for .git/objects/pack
I can suggest the minimal change for your setup.oleander-whirlwind
10/09/2025, 9:02 AMMarvin
10/09/2025, 9:02 AMMarvin
10/09/2025, 9:03 AMflow.from_source(...).serve(...) works. What you’re hitting is a stale destination with mismatched permissions/ownership, so subsequent pulls try to overwrite .git/objects/pack/* and fail with EACCES. That can happen if a prior run created those files with a different user (e.g., root) or with read-only perms. It’s not inherent to serve; it’s the reuse of an existing destination with files you can’t overwrite.
Why you see this
- The runner clones to a staging tmp dir, then copies into a “runner storage” destination. If that destination (or files within, like .git/objects/pack/*) already exist with restrictive perms or different ownership, a second pull/copy fails.
- This often occurs if you ran once as root and later as a non-root user, or if you pointed the destination out of /tmp to a path owned by a different UID.
What to do
Pick one of these approaches:
1) One-time cleanup to clear stale, mis-owned files
rm -rf ${TMPDIR:-/tmp}/runner_storage/*
# If some dirs are root-owned, run with sudo or fix ownership:
# sudo chown -R "$(id -u):$(id -g)" ${TMPDIR:-/tmp}/runner_storage
Then re-run your script. This usually resolves the immediate error.
2) Ensure a fresh, user-owned temp base and clean at startup
Set TMPDIR before importing Prefect so runner storage lands in a path you control, and clean it on startup:
import os, shutil
from pathlib import Path
os.environ["TMPDIR"] = "/workspaces/your-repo/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
# Optional: proactively clear prior runner storage for a clean start
shutil.rmtree(Path(os.environ["TMPDIR"]) / "runner_storage", ignore_errors=True)
3) Exclude .git from what’s copied
If your GitLab repository block supports selecting directories, only copy your code dirs so .git is not included:
gitlab_repository_block.directories = ["workflows"] # adjust to your code dirs
This avoids overwriting .git pack files entirely.
4) Pre-clone yourself and serve from that path
If you want full control, clone to a fresh directory you own and serve from there:
from prefect import flow as flow_cls
local_path = "/workspaces/your-repo/_src"
# ...git clone or clean this path first...
f = flow_cls.from_source(
source=local_path,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
How to validate the checkout step without serving
You can preflight the clone/pull and see exactly where Prefect writes:
from prefect.runner.storage import BlockStorageAdapter
from prefect_gitlab import GitLabRepository, GitLabCredentials
import asyncio
async def main():
repo = GitLabRepository.load("prefectprototypegit")
repo.credentials = GitLabCredentials.load("mycredential")
storage = BlockStorageAdapter(block=repo)
await storage.pull_code()
print("Pulled to:", storage.destination)
asyncio.run(main())
Run it twice; if the second run fails, it confirms reuse of an existing destination with files you can’t overwrite. Cleaning or changing TMPDIR should fix it.
Next step debug
Please share:
- Ownership of the failing path (this will confirm a root vs non-root mismatch):
ls -l /uvd-workflow-orchestrator-example-prefect/gitlab-repository-prefectprototypegit/.git/objects/pack
- Whether you previously ran the flow as root inside the container.
Refs
- How Prefect stores/loads code from source: Store flow code for deployments
- Related context on runner temp storage: Issue discussionoleander-whirlwind
10/09/2025, 10:37 AMMarvin
10/09/2025, 10:37 AMMarvin
10/09/2025, 10:39 AMrepo = GitLabRepository.load("prefectprototypegit")
repo.credentials = GitLabCredentials.load("mycredential")
repo.directories = ["workflows"] # adjust to your code dir(s)
f = flow.from_source(
source=repo,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
This is the cleanest fix because it avoids overwriting .git pack files entirely.
Option 2 — Disable periodic pulls for the session
If you don’t need auto-refreshing code while serving, disable the pull loop by wrapping your block with BlockStorageAdapter and setting pull_interval=None:
from prefect.runner.storage import BlockStorageAdapter
repo = GitLabRepository.load("prefectprototypegit")
repo.credentials = GitLabCredentials.load("mycredential")
storage = BlockStorageAdapter(block=repo, pull_interval=None) # no periodic re-copy
f = flow.from_source(
source=storage,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
Option 3 — Pre-clean runner_storage at startup
If you want periodic pulls but are hitting stale files, clear the runner destination before serve starts. Set TMPDIR to a path you own and remove the runner_storage dir:
import os, shutil, tempfile
from pathlib import Path
os.environ["TMPDIR"] = "/uvd-workflow-orchestrator-example-prefect/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
shutil.rmtree(Path(tempfile.gettempdir()) / "runner_storage", ignore_errors=True)
# then construct your repo/block and serve as usual
Option 4 — Keep your working pre-download pattern
Clone to a fresh directory you own (or clean it first), then serve from that local path:
f = flow.from_source(
source="/uvd-workflow-orchestrator-example-prefect/_src", # your own clone target
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
Why this happens
- Serve uses a fresh UUID destination under ${TMPDIR:-/tmp}/runner_storage.
- On each pull_interval (default 60s), it copies again into the same destination.
- .git/objects/pack files from the first copy are typically 0444; the second copy tries to overwrite them and gets EACCES.
I recommend Option 1 (directories=["workflows"]) if your block supports it; otherwise Option 2 (pull_interval=None) is a quick workaround. If you want, paste your GitLabRepository block’s attribute list and I can confirm the directories setting name for that block.
Refs:
- Store flow code from source: How-to: store flow code from source
- Background on runner storage behavior: Related discussionoleander-whirlwind
10/09/2025, 11:20 AMMarvin
10/09/2025, 11:20 AMMarvin
10/09/2025, 11:22 AMimport os
from prefect import flow as flow_cls
from prefect.runner.storage import GitRepository
# Example using a basic HTTPS URL with a token in env (avoid committing tokens!)
# For GitLab, username can be anything; many use 'oauth2' or your actual username.
GITLAB_TOKEN = os.environ["GITLAB_TOKEN"]
url = f"<https://oauth2:{GITLAB_TOKEN}@gitlab.com/your-group/your-repo.git>"
repo = GitRepository(
url=url,
directories=["workflows"], # only copy your flow code; .git is not copied
pull_interval=60, # or None to disable periodic pulls
)
f = flow_cls.from_source(
source=repo,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
- This avoids the shutil EACCES on .git/objects/pack during the periodic copy.
B) Keep GitLabRepository but avoid periodic re-copying
- Wrap your block with BlockStorageAdapter and disable the periodic pull that triggers the overwrite of .git pack files:
from prefect.runner.storage import BlockStorageAdapter
from prefect_gitlab import GitLabRepository, GitLabCredentials
from prefect import flow as flow_cls
repo = GitLabRepository.load("prefectprototypegit")
repo.credentials = GitLabCredentials.load("mycredential")
storage = BlockStorageAdapter(block=repo, pull_interval=None) # no periodic re-copy
f = flow_cls.from_source(
source=storage,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
- This keeps the GitLab block but prevents the second copy (which is when .git overwrite fails). If you still want periodic updates, you can implement a manual refresh step between restarts or set a long pull_interval to reduce churn.
Why directories matters here
- The failure occurs when serve copies from a staging clone to the runner_storage destination. On the second copy, .git/objects/pack files from the first copy are read-only and can’t be overwritten. Excluding .git avoids this entirely, but that capability is only present on core GitRepository.
Useful links
- Store flow code from source: How-to: store flow code from source
- prefect-gitlab integration overview: prefect-gitlab integration
If you want, paste your GitLab URL pattern (SSH vs HTTPS), and I’ll tailor the GitRepository example with the right auth pattern for your setup.oleander-whirlwind
10/09/2025, 11:35 AMMarvin
10/09/2025, 11:35 AMMarvin
10/09/2025, 11:36 AMrm -rf "/uvd-workflow-orchestrator-example-prefect/.tmp/runner_storage/b2f7289f-38dc-47a8-9fb4-ac790676347d/uvd-workflow-orchestrator-example-prefect.git"
- Or clean all runner storage (safe in dev):
rm -rf "/uvd-workflow-orchestrator-example-prefect/.tmp/runner_storage"
Make it robust at startup
- Set TMPDIR and pre-clean runner_storage before importing Prefect, so you always start fresh:
import os, shutil, tempfile
from pathlib import Path
os.environ["TMPDIR"] = "/uvd-workflow-orchestrator-example-prefect/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
# Clear any stale repos to avoid remote mismatch / EACCES on .git/objects/pack
shutil.rmtree(Path(tempfile.gettempdir()) / "runner_storage", ignore_errors=True)
Then keep your Option B pattern:
from prefect.runner.storage import BlockStorageAdapter
from prefect_gitlab import GitLabRepository, GitLabCredentials
from prefect import flow as flow_cls
repo = GitLabRepository.load("prefectprototypegit")
repo.credentials = GitLabCredentials.load("mycredential")
storage = BlockStorageAdapter(block=repo, pull_interval=None) # no periodic re-copy
f = flow_cls.from_source(
source=storage,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
If you want to diagnose the mismatch before deleting, inspect the existing remote:
git -C "/uvd-workflow-orchestrator-example-prefect/.tmp/runner_storage/b2f7289f-38dc-47a8-9fb4-ac790676347d/uvd-workflow-orchestrator-example-prefect.git" remote -v
Compare with your GitLabRepository’s configured URL.
Alternative that avoids these .git issues entirely
- Use the core GitRepository (supports directories) and only copy your code directory (not .git). This avoids both read-only pack files and remote mismatches:
import os
from prefect import flow as flow_cls
from prefect.runner.storage import GitRepository
os.environ["TMPDIR"] = "/uvd-workflow-orchestrator-example-prefect/.tmp"
# Use HTTPS with a token in env for GitLab
url = f"<https://oauth2:{os.environ['GITLAB_TOKEN']}@gitlab.com/your-group/your-repo.git>"
repo = GitRepository(
url=url,
directories=["workflows"], # only copy your flow code; .git is not copied
pull_interval=60, # or None if you don't want periodic refresh
)
f = flow_cls.from_source(
source=repo,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
Two final notes
- You’re on Python 3.13; Prefect 3.x is tested on 3.10–3.12. If you still see odd asyncio/serve behavior after cleaning runner_storage, try 3.12.
- Docs for storing code from source: Store flow code from source
If you share the output of the git remote -v command for that path, I can confirm whether it’s a real mismatch or just a stale folder collision.oleander-whirlwind
10/09/2025, 11:43 AMMarvin
10/09/2025, 11:43 AMMarvin
10/09/2025, 11:43 AMrepo.credentials = GitLabCredentials.load("mycredential") only mutates the in-memory object. If you don’t save it back, the block in the API still has no credentials, so the subprocess sees credentials=None and fails.
Fix: persist the credentials on the block before serving
Do this once, then serve. Make sure you do this before importing/using the block for from_source.
from prefect_gitlab import GitLabRepository, GitLabCredentials
# Ensure your credentials block exists and has the token
creds = GitLabCredentials.load("mycredential") # or create and save it if needed
# Attach creds to the repo block and SAVE it so the API has them
repo = GitLabRepository.load("prefectprototypegit")
repo.credentials = creds
repo.save("prefectprototypegit", overwrite=True)
# Sanity check: new load should show credentials present
assert GitLabRepository.load("prefectprototypegit").credentials is not None
Then use the saved block with BlockStorageAdapter (to avoid the .git pack overwrite) or use it directly:
import os
from pathlib import Path
from prefect import flow as flow_cls
from prefect.runner.storage import BlockStorageAdapter
from prefect_gitlab import GitLabRepository
# Optional but recommended: control temp base and start clean
os.environ["TMPDIR"] = "/uvd-workflow-orchestrator-example-prefect/.tmp"
Path(os.environ["TMPDIR"]).mkdir(parents=True, exist_ok=True)
repo = GitLabRepository.load("prefectprototypegit")
# Disable periodic recopy to avoid .git pack overwrites
storage = BlockStorageAdapter(block=repo, pull_interval=None)
f = flow_cls.from_source(
source=storage,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
Alternative: use core GitRepository (supports directories)
If you’d rather avoid copying .git entirely, switch to Prefect’s core GitRepository and copy only your code directory:
from prefect.runner.storage import GitRepository
from prefect import flow as flow_cls
import os
os.environ["TMPDIR"] = "/uvd-workflow-orchestrator-example-prefect/.tmp"
url = f"<https://oauth2:{os.environ['GITLAB_TOKEN']}@gitlab.com/your-group/your-repo.git>"
repo = GitRepository(
url=url,
directories=["workflows"], # only copy your flow code; .git not copied
pull_interval=60, # or None to disable periodic refresh
)
f = flow_cls.from_source(
source=repo,
entrypoint="workflows/workflow_with_non_and_decorated_task.py:my_flow",
)
f.serve(name="private-gitlab-deploy")
Two additional tips
- Clean runner storage between experiments to avoid stale repos/permissions:
rm -rf "/uvd-workflow-orchestrator-example-prefect/.tmp/runner_storage"
- You’re on Python 3.13; Prefect 3.x is tested on 3.10–3.12. If you see unrelated asyncio issues, try 3.12 to rule that out.
Docs
- GitLab integration overview: prefect-gitlab
- Storing/serving flow code from source: Store flow code from source
If you run the assert check and it still shows credentials missing, paste the output of:
from prefect_gitlab import GitLabRepository
r = GitLabRepository.load("prefectprototypegit")
print("Has creds?", r.credentials is not None)
and we’ll sort out the block contents.