Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hello! I have more and more issue like this one: `prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect.deployments.steps.set_working_directory` even if the directory exists (If I re-run the job multiple times, it starts to work). Is it a known issue?

My `prefect.yaml` is like this:
```name: metabase-reports
prefect-version: 2.14.17

pull:
- prefect.deployments.steps.set_working_directory:
    directory: "/usr/lib/metabase-reports/prefect/"```

hi <@U0406K6K2CA> its always helpful if you can provide the whole trace so we can see how we're getting to that eventual error

Yes, sorry :
```Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 154, in run_steps
    step_output = await run_step(step, upstream_outputs)
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 125, in run_step
    result = await from_async.call_soon_in_new_thread(
  File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 293, in aresult
    return await asyncio.wrap_future(self.future)
  File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 318, in _run_sync
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments/steps/pull.py", line 28, in set_working_directory
    os.chdir(directory)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/metabase-reports/prefect/'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 425, in retrieve_flow_then_begin_flow_run
    else await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments/deployments.py", line 249, in load_flow_from_flow_run
    output = await run_steps(deployment.pull_steps)
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 182, in run_steps
    raise StepExecutionError(f"Encountered error while running {fqn}") from exc
prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect.deployments.steps.set_working_directory```

thanks! hmm what is the pull policy on your (i'm assuming) k8s work pool?

image.png

Yes, it's a Kubernetes Work Pool :slightly_smiling_face: The Pull Policy is `IfNotPresent`