David Hlavaty
09/01/2022, 12:33 PMpython:3-9slim
and I then install all my dependencies including Prefect from a lock file. I prefer this over using the official Prefect images as it means there is a single source of truth for which version of Prefect I am using.
I had my image "misconfigured" by setting working directory to /opt/prefect/flows
. This then resulted in a cryptic message from shutil
when running my flows as it tried to copy files from the flow source directory ( /opt/prefect/flows
by default) to a working directory (/opt/prefect/flows
) which are the same.
Would be good if Prefect detected this and did not attempt to copy the files. Or at least checked if the working directory is the same and failed with helpful error message.Adam Brusselback
09/01/2022, 12:54 PMDavid Hlavaty
09/01/2022, 4:03 PMsrc dir == dest dir
In your case, the problem originates from using Python 3.7 as behaviour of https://docs.python.org/3.7/library/shutil.html#shutil.copytree
Recursively copy an entire directory tree rooted at src, returning the destination directory. The destination directory, named by dst, must not already exist; it will be created as well as missing parent directories. Permissions and times of directories are copied withIf you follow stack-trace from your issue, it will lead you here, individual files are copied usingcopystat()
.shutil.copy2()
if sys.version_info < (3, 8):
shutil.copytree(from_path, local_path)
else:
shutil.copytree(from_path, local_path, dirs_exist_ok=True)
Notice that for Python 3.7 and lower it does not tolerate existing directory (it can't as per docs above). Given that local_path
default to current working directory, I don't think this can ever work since the current working directory exists by definition.Adam Brusselback
09/01/2022, 4:52 PM