Hi, I'm trying to work through the Prefect Orion t...
# prefect-community
Hi, I'm trying to work through the Prefect Orion tutorial for running flows in Kubernetes (https://orion-docs.prefect.io/tutorials/kubernetes-flow-runner/) and I keep running into the same problem. When I run the deployment,
prefect deployment run test-flow/test-deployment
, the UI shows that the flow run is Pending. When I look at the logs in Kubernetes with
kubectl logs -l job-name=voracious-antelope2b89g
, I see a FileNotFound stack trace Thank you for your help!
Hi @James Harr, I might have time to try to replicate this later but if I don’t, I will surely ask some of the engineers on Monday. Would appreciate it if you could move the traceback to this thread when you get a chance to keep the main channel cleaner
👍 1
No problem, thank you for taking a look!
Copy code
async with await anyio.open_file(storage_path, mode="rb") as fp:
  File "/usr/local/lib/python3.9/site-packages/anyio/_core/_fileio.py", line 156, in open_file
    fp = await to_thread.run_sync(open, file, mode, buffering, encoding, errors, newline,
  File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/0q/dynsm9h90vjb8py9gz9m0q8c0000gn/T/prefect/f000f05a-ab00-4b66-b673-aed9dfb57bef'
When I look at the Deployments tab in the UI, I can see my deployment with some summary text that says:
Copy code
  "data": "\"/var/folders/0q/dynsm9h90vjb8py9gz9m0q8c0000gn/T/prefect/f000f05a-ab00-4b66-b673-aed9dfb57bef\"",
  "block_id": null
The value for
is the same location that the pod logs say cannot be found. My guess is that my
file is not being copied into the container that the pod runs. Also, when I create the deployment I get a warning:
Copy code
.../prefect/client.py:1227: UserWarning: No default storage has been set on the server. Using temporary local storage for results.
Is this part of the problem also? It's using my local machine to store the deployment info but then when deploying into Kubernetes it doesn't have access to that file?
👍 1
🤔 1
Thank you!
Basically yeah you need to use non-local storage for the deployment to be retrievable from K8s
We’ve got updated tutorials coming very soon and a more intuitive UX for this on the way as well.