https://prefect.io logo
m

Michael Wedekindt

06/09/2021, 4:26 PM
Hi, I have trouble to run a DbtShellTask on a Windows machine When I run this
Copy code
with Flow(name="dbt_flow") as f:
    task = DbtShellTask(
        log_stderr=True,
        log_stdout=True,
        return_all=True,
        stream_output=True,
        profiles_dir=profile_path

    )(command='dbt run --models data_hub')

out = f.run()
I got this error message back
Copy code
[2021-06-09 16:06:15+0200] INFO - prefect.FlowRunner | Beginning Flow run for 'dbt_flow'
[2021-06-09 16:06:15+0200] INFO - prefect.TaskRunner | Task 'DbtShellTask': Starting task run...
[2021-06-09 16:06:15+0200] INFO - prefect.DbtShellTask | /bin/bash: C:\Users\M7856~1.\AppData\Local\Temp\prefect-5yoqhqg_: No such file or directory
[2021-06-09 16:06:15+0200] ERROR - prefect.DbtShellTask | Command failed with exit code 127
[2021-06-09 16:06:15+0200] INFO - prefect.TaskRunner | FAIL signal raised: FAIL('Command failed with exit code 127')
[2021-06-09 16:06:15+0200] INFO - prefect.TaskRunner | Task 'DbtShellTask': Finished task run for task with final state: 'Failed'
[2021-06-09 16:06:15+0200] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
When I try this with the common ShellTask I get the same error. Here some background facts: I already added bash to path and I can open a bash console I use Visual Studio Code for developing My Python Version is 3.9.5 My prefect Version is 0.14.21 Usually I run my dbt locally in the Ubuntu Subsystem on Windows but could run dbt in windows cmd as well Thanks in advance for help and best regards, Michael
z

Zach Schumacher

06/09/2021, 4:33 PM
you need to change your shell
Copy code
def __init__(
        self,
        command: str = None,
        env: dict = None,
        helper_script: str = None,
        shell: str = "bash",
        return_all: bool = False,
        log_stderr: bool = False,
        stream_output: Union[bool, int, str] = False,
        **kwargs: Any,
    )
m

Michael Wedekindt

06/09/2021, 4:42 PM
I tried this as well and set
shell = 'cmd'
but then Its waiting for input from my site, but I cant exec anything instead
dir
then it lists folder content and exit. are there more options?
k

Kevin Kho

06/09/2021, 4:57 PM
I’ll try this on a windows machine later today
m

Michael Wedekindt

06/09/2021, 4:57 PM
I tried
shell="powershell"
then I get this messages
Copy code
[2021-06-09 18:55:12+0200] INFO - prefect.FlowRunner | Beginning Flow run for 'dbt_flow'
[2021-06-09 18:55:13+0200] INFO - prefect.TaskRunner | Task 'DbtShellTask': Starting task run...
[2021-06-09 18:55:13+0200] INFO - prefect.TaskRunner | Task 'DbtShellTask': Finished task run for task with final state: 'Success'
[2021-06-09 18:55:13+0200] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded
But I dont see if he has executed dbt or at least try to execute
z

Zach Schumacher

06/09/2021, 5:20 PM
and you have stream_output=True when you did that?
k

Kevin Kho

06/09/2021, 7:09 PM
Hey I didn’t get a chance to try this yet, but this Github comment may provide additional details to help you
m

Michael Wedekindt

06/15/2021, 3:28 PM
Hey Kevin, thanks I was not able the last days to have a look as well 😄 I'll will try this and let the community know 👍
k

Kevin Kho

06/15/2021, 3:29 PM
Sounds good!