Yossi
06/18/2023, 2:58 PMset_working_directory
step in Prefect deployments.
In the prefect.deployments.steps.pull.set_working_directory function, the code executes the os.chdir(directory)
command to change the working directory. Then, it returns a dictionary with the key directory
set to the specified directory.
In the deployments.load_flow_from_flow_run function, if a deployment includes pull steps, it first runs the steps using the run_steps
function. Then, it checks if the output dictionary contains the key directory
by executing if output.get("directory")
. If the key is present, it attempts to change the directory.
I guess the issue is clear. It arises when there is a set_working_directory
step in the deployment. Initially, the working directory is changed within the set_working_directory
step. However, load_flow_from_flow_run
continue and tries to change the directory again based on the directory key in the output variable. This results in a failure because attempting to change the directory in the new current directory causes a FileNotFoundError
with the message [Errno 2] No such file or directory: '<user-directory-argument>'
.
I would be happy to understand if I'm wrong here since I guess I'm not the first one who tried to use this step, but it crushed on my setup and from the code it seems to be clear that this can not work.
Thanks in advanced!alex
06/19/2023, 1:47 PMset_working_directory
step only work with absolute paths for the exact reasons that you mentioned. If you would like to use set_working_directory
with relative paths, feel free to open an issue and we can explore solutions.