Tim-Oliver
11/23/2022, 4:59 PMPermission denied
errors with the github storage blocks since today. Has anyone seen this as well? The error appears if the github repo is already present from a previous run. If the repo is deleted the gitrepo is cloned without any error.shutil.Error: [('/tmp/tmpzw4y7qtmprefect/.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f', './.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f', "[Errno 13] Permission denied: './.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97', './.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97', "[Errno 13] Permission denied: './.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786', './.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786', "[Errno 13] Permission denied: './.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97', './.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97', "[Errno 13] Permission denied: './.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391', './.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391', "[Errno 13] Permission denied: './.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261', './.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261', "[Errno 13] Permission denied: './.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529', './.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529', "[Errno 13] Permission denied: './.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/9d/10168d3633499e25522a57c8109767f0819735', './.git/objects/9d/10168d3633499e25522a57c8109767f0819735', "[Errno 13] Permission denied: './.git/objects/9d/10168d3633499e25522a57c8109767f0819735'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/3e/5551707d7888df29e5182461fcc324efeda522', './.git/objects/3e/5551707d7888df29e5182461fcc324efeda522', "[Errno 13] Permission denied: './.git/objects/3e/5551707d7888df29e5182461fcc324efeda522'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634', './.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634', "[Errno 13] Permission denied: './.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da', './.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da', "[Errno 13] Permission denied: './.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/f9/53d1d075afbc06216855d7855503f9ea9b70a0', './.git/objects/f9/53d1d075afbc06216855d7855503f9ea9b70a0', "[Errno 13] Permission denied: './.git/objects/f9/53d1d075afbc06216855d7855503f9ea9b70a0'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da', './.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da', "[Errno 13] Permission denied: './.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42', './.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42', "[Errno 13] Permission denied: './.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6', './.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6', "[Errno 13] Permission denied: './.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6'"), ('/tmp/tmpzw4y7qtmprefect/.git/objects/be/e211a193292471d39e45698d228ee54039d802', './.git/objects/be/e211a193292471d39e45698d228ee54039d802', "[Errno 13] Permission denied: './.git/objects/be/e211a193292471d39e45698d228ee54039d802'")]
Anna Geller
11/23/2022, 8:00 PMTim-Oliver
11/24/2022, 7:29 AMprefect-working-dir/.git/objects
have only read only permissions.Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/engine.py", line 255, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/deployments.py", line 163, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/filesystems.py", line 917, in get_directory
copytree(src=content_source, dst=content_destination, dirs_exist_ok=True)
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/shutil.py", line 568, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/shutil.py", line 522, in _copytree
raise Error(errors)
shutil.Error: [('/tmp/tmpjf2bug7lprefect/.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f', './.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f', "[Errno 13] Permission denied: './.git/objects/51/16fdcab0c85837d53ac9ee05bbeb7b0831429f'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97', './.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97', "[Errno 13] Permission denied: './.git/objects/b7/6ed055126028e448438c6c1503991c8a70de97'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786', './.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786', "[Errno 13] Permission denied: './.git/objects/d5/222fd9a29067934daed5b4b0fb6307f70cf786'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97', './.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97', "[Errno 13] Permission denied: './.git/objects/f4/0a470220a9ccc37f0005f7978e8df1695dfb97'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391', './.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391', "[Errno 13] Permission denied: './.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261', './.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261', "[Errno 13] Permission denied: './.git/objects/35/9b656038f5c35061d26f5f73cdd258d29d0261'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529', './.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529', "[Errno 13] Permission denied: './.git/objects/15/9200a08fdc5741d070d432338b9c2ef18ed529'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/9d/10168d3633499e25522a57c8109767f0819735', './.git/objects/9d/10168d3633499e25522a57c8109767f0819735', "[Errno 13] Permission denied: './.git/objects/9d/10168d3633499e25522a57c8109767f0819735'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/3e/5551707d7888df29e5182461fcc324efeda522', './.git/objects/3e/5551707d7888df29e5182461fcc324efeda522', "[Errno 13] Permission denied: './.git/objects/3e/5551707d7888df29e5182461fcc324efeda522'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634', './.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634', "[Errno 13] Permission denied: './.git/objects/b2/c164d3a81663f1823a1baebc1b98f589fc8634'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da', './.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da', "[Errno 13] Permission denied: './.git/objects/62/dc4b9d007c19610d4c8cf9b43b7851c2b315da'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/fc/09abcf4631f147623fcd3f0f1cd9f9af36cb7c', './.git/objects/fc/09abcf4631f147623fcd3f0f1cd9f9af36cb7c', "[Errno 13] Permission denied: './.git/objects/fc/09abcf4631f147623fcd3f0f1cd9f9af36cb7c'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/a8/c3a0a8b0be94c400fde72a6b045c8759c5167f', './.git/objects/a8/c3a0a8b0be94c400fde72a6b045c8759c5167f', "[Errno 13] Permission denied: './.git/objects/a8/c3a0a8b0be94c400fde72a6b045c8759c5167f'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da', './.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da', "[Errno 13] Permission denied: './.git/objects/3a/d797acf2f668e46454014df230ce4a5a76b1da'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42', './.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42', "[Errno 13] Permission denied: './.git/objects/79/c899a064ac6800264c61d9cfb04104df966a42'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6', './.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6', "[Errno 13] Permission denied: './.git/objects/cf/b8a3d7361b654ac455084b181cadd7a5f3c4e6'"), ('/tmp/tmpjf2bug7lprefect/.git/objects/be/e211a193292471d39e45698d228ee54039d802', './.git/objects/be/e211a193292471d39e45698d228ee54039d802', "[Errno 13] Permission denied: './.git/objects/be/e211a193292471d39e45698d228ee54039d802'")]
rwxrwxr-x
. Then I run the flow and the permissions change to rwx------
. The next run fails due to permission denied on copying the git-repo contents from tmp to the working dir.Anna Geller
11/24/2022, 3:04 PMTim-Oliver
11/24/2022, 3:09 PMPeyton Runyan
11/24/2022, 3:34 PMTim-Oliver
11/24/2022, 3:39 PMPeyton Runyan
11/24/2022, 3:48 PMdistutils
implementation of copy_tree
to shutil.copytree
because distutils
is scheduled to be deprecated. I'm wondering if this caused a regression.
https://github.com/PrefectHQ/prefect/pull/7441Tim-Oliver
11/24/2022, 3:51 PMPeyton Runyan
11/24/2022, 3:53 PMsrc/prefect/filesystems.py
and delete this import:
from prefect.utilities.compat import copytree
and add this import:
from distutils.dir_util import copy_tree as copytree
?Tim-Oliver
11/24/2022, 4:10 PMFlow could not be retrieved from deployment.
Traceback (most recent call last):
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/engine.py", line 255, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/deployments.py", line 163, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/tungstenfs/scratch/gmicro_share/_prefect/miniconda3/envs/test_gfriedri-em-alignment-flows/lib/python3.9/site-packages/prefect/filesystems.py", line 917, in get_directory
copytree(src=content_source, dst=content_destination, dirs_exist_ok=True)
TypeError: copy_tree() got an unexpected keyword argument 'dirs_exist_ok'
Peyton Runyan
11/24/2022, 4:17 PM917
and change the signature to
copytree(src=content_source, dst=content_destination)
and then try again?Tim-Oliver
11/24/2022, 4:22 PMPeyton Runyan
11/28/2022, 1:05 PMTim-Oliver
11/28/2022, 1:10 PM