Trying to use DbtShellTask on windows but get the following error:
Copy code
[2022-06-27 17:16:15+0200] ERROR - prefect.DbtShellTask | /bin/bash: C:UsersUSERAD~1AppDataLocalTempprefect-6rbqzzlx: No such file or directory
This line in prefect.tasks.shell.py generates a strange path:
Copy code
with tempfile.NamedTemporaryFile(prefix="prefect-") as tmp:
Andreas Nord
06/28/2022, 7:58 AM
Running something simple with ShellTask
Copy code
from prefect import Flow
from prefect.tasks.shell import ShellTask
task = ShellTask(log_stderr=True)
with Flow("My Flow") as f:
# both tasks will be executed in home directory
contents = task(command='ls')
out = f.run()
gives the same error:
Copy code
[2022-06-28 09:55:31+0200] INFO - prefect.FlowRunner | Beginning Flow run for 'My Flow'
[2022-06-28 09:55:31+0200] INFO - prefect.TaskRunner | Task 'ShellTask': Starting task run...
[2022-06-28 09:55:31+0200] ERROR - prefect.ShellTask | Command failed with exit code 127
[2022-06-28 09:55:31+0200] ERROR - prefect.ShellTask | /bin/bash: C:UsersUSERAD~1AppDataLocalTempprefect-_exffnpu: No such file or directory
✅ 1
k
Kevin Kho
06/28/2022, 8:00 AM
This looks like bash is not in the Windows path?
a
Andreas Nord
06/28/2022, 8:02 AM
My interpretation is that this file does not exist:
C:UsersUSERAD~1AppDataLocalTempprefect-_exffnpu
k
Kevin Kho
06/28/2022, 8:02 AM
Oh not even, the path out of your TemporaryFile looks weird
Kevin Kho
06/28/2022, 8:03 AM
I guess you could use the
os.subprocess
call inside a task
a
Andreas Nord
06/28/2022, 8:07 AM
Yes not sure what is the issue with tempfile behaviour, will have to solve it with os.subprocess
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.