https://prefect.io logo
Join Slack
Powered by
Also, I am having other (file permissions?) failur...
# ask-community
a
Also, I am having other (file permissions?) failures when trying to run on my deployed infrastructure rather than locally for the exact same flows mentioned above (see log in thread) I wasn't able to find anything pointing me to a resolution here... the files exist in the specified deploy location, they work in my local docker container, they don't work on this VM.
✅ 1
Log:
Copy code
Sep  1 01:41:12 check-prefectagent01 prefect[14263]: 01:41:12.436 | INFO    | prefect.agent - Submitting flow run '49507f89-c9e5-49b5-8b28-4e522f9f4253'
Sep  1 01:41:14 check-prefectagent01 prefect[14263]: 01:41:14.575 | INFO    | prefect.infrastructure.process - Opening process 'abstract-roadrunner'...
Sep  1 01:41:14 check-prefectagent01 prefect[14263]: 01:41:14.585 | INFO    | prefect.agent - Completed submission of flow run '49507f89-c9e5-49b5-8b28-4e522f9f4253'
Sep  1 01:41:16 check-prefectagent01 prefect[14263]: 01:41:16.336 | ERROR   | Flow run 'abstract-roadrunner' - Flow could not be retrieved from deployment.
Sep  1 01:41:16 check-prefectagent01 prefect[14263]: Traceback (most recent call last):
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:   File "/usr/local/lib/python3.7/dist-packages/prefect/engine.py", line 256, in retrieve_flow_then_begin_flow_run
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:     flow = await load_flow_from_flow_run(flow_run, client=client)
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:   File "/usr/local/lib/python3.7/dist-packages/prefect/client.py", line 103, in with_injected_client
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:     return await fn(*args, **kwargs)
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:   File "/usr/local/lib/python3.7/dist-packages/prefect/deployments.py", line 54, in load_flow_from_flow_run
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:     await storage_block.get_directory(from_path=deployment.path, local_path=".")
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:   File "/usr/local/lib/python3.7/dist-packages/prefect/filesystems.py", line 132, in get_directory
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:     shutil.copytree(from_path, local_path)
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:   File "/usr/lib/python3.7/shutil.py", line 321, in copytree
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:     os.makedirs(dst)
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:   File "/usr/lib/python3.7/os.py", line 221, in makedirs
Sep  1 01:41:16 check-prefectagent01 prefect[14263]:     mkdir(name, mode)
Sep  1 01:41:16 check-prefectagent01 prefect[14263]: FileExistsError: [Errno 17] File exists: '.'
Sep  1 01:41:16 check-prefectagent01 prefect[14263]: 01:41:16.648 | INFO    | prefect.infrastructure.process - Process 'abstract-roadrunner' exited cleanly.
Looks like the difference is my VM that is having the issue is running Debian 10 and Python 3.7, and the Docker container is running Python 3.9.
I then saw this in the filesystems.py get_directory() function as a difference:
Copy code
if sys.version_info < (3, 8):
    shutil.copytree(from_path, local_path)
else:
    shutil.copytree(from_path, local_path, dirs_exist_ok=True)
And looking further, the docs say I should be able to set ignore_storage=True and things should be fine... that's not possible from the code I see though: https://docs.prefect.io/api-ref/prefect/deployments/?h=#prefect.deployments.load_flow_from_flow_run
✅ 1
j
Hi Adam. Thanks for the info. Sounds like you might have found a bug. I looked at the issues and didn’t see an open one on the topic. Are you able to open an issue with an MRE over at https://github.com/PrefectHQ/prefect/issues/6641. Also, in an effort to keep the main channel tidy are you able to please put the logs/traceback in the thread?
a
Moved it, will open another issue.
👍 1
🙏 1
2 Views
Open in Slack
PreviousNext
Powered by