James Harr
03/12/2022, 2:20 PMprefect deployment run test-flow/test-deployment
, the UI shows that the flow run is Pending. When I look at the logs in Kubernetes with kubectl logs -l job-name=voracious-antelope2b89g
, I see a FileNotFound stack trace
Thank you for your help!Kevin Kho
03/12/2022, 7:10 PMJames Harr
03/12/2022, 8:06 PMasync with await anyio.open_file(storage_path, mode="rb") as fp:
File "/usr/local/lib/python3.9/site-packages/anyio/_core/_fileio.py", line 156, in open_file
fp = await to_thread.run_sync(open, file, mode, buffering, encoding, errors, newline,
File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 28, in run_sync
return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 754, in run
result = context.run(func, *args)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/0q/dynsm9h90vjb8py9gz9m0q8c0000gn/T/prefect/f000f05a-ab00-4b66-b673-aed9dfb57bef'
When I look at the Deployments tab in the UI, I can see my deployment with some summary text that says:
{
"data": "\"/var/folders/0q/dynsm9h90vjb8py9gz9m0q8c0000gn/T/prefect/f000f05a-ab00-4b66-b673-aed9dfb57bef\"",
"block_id": null
}
The value for data
is the same location that the pod logs say cannot be found. My guess is that my k8s_flow.py
file is not being copied into the container that the pod runs.
Also, when I create the deployment I get a warning:
.../prefect/client.py:1227: UserWarning: No default storage has been set on the server. Using temporary local storage for results.
Is this part of the problem also? It's using my local machine to store the deployment info but then when deploying into Kubernetes it doesn't have access to that file?Kevin Kho
03/12/2022, 8:16 PMZanie
03/14/2022, 1:51 PM