https://prefect.io logo
Title
c

Carlos Cueto

02/17/2023, 4:44 PM
Hi there. I've been trying to migrate all of the flows that use
shell_run_command
from
prefect-shell
into the new
ShellOperation
class, however, I'm always running into the following error when I try to run ANY command. This is using WINDOWS. Examples:
ShellOperation(commands=["echo Hello"]).run()
1
m

Mansour Zayer

02/17/2023, 5:31 PM
From my own experience, I suggest using python's built-in
subprocess
module, especially if you're on windows. You can run a shell task like
import subprocess

subprocess.run(command, stdout=True, check=True)
Under the hood, the shell_run_command and shellOperation also use this module, but I think the problems that come with them aren't worth the trouble.
n

Nate

02/17/2023, 6:58 PM
Hi @Carlos Cueto - thanks for raising this! this looks like a windows-specific issue with how we write commands to a temp dir before executing them, we'll explore this more! and just to clarify @Mansour Zayer, we do not use subprocess.run, we use anyio.open_process
🙌 1
m

Mansour Zayer

02/17/2023, 6:59 PM
Thanks for the clarification. I think subprocess was used in prefect1 shell tasks (or I might just be mistaken all together)
n

Nate

02/17/2023, 7:01 PM
(or I might just be mistaken all together)
nope! you're totally right, we did use subprocess in v1 version of shell task
c

Carlos Cueto

02/17/2023, 7:07 PM
Thank you @Nate. I'm surprised that I'm the only one that has reported this. Maybe not a lot of people use Windows agents with local processes and
prefect-shell
@Nate BTW, I opened an issue under the
prefect-shell
GitHub as well: https://github.com/PrefectHQ/prefect-shell/issues/69
:thank-you: 1
a

Andrew Huang

02/17/2023, 7:36 PM
@Carlos Cueto I created a branch https://github.com/PrefectHQ/prefect-shell/pull/70 do you mind trying to install this and rerunning? also can you let us know which Python version you’re using?
pip install git+<https://github.com/PrefectHQ/prefect-shell.git@flush_after_write>
c

Carlos Cueto

02/17/2023, 7:44 PM
@Andrew Huang Installed the branch version you gave me. Running on python 3.10.7, but error also occurs in another machine with 3.9.2. Getting following error now:
a

Andrew Huang

02/17/2023, 7:47 PM
Thanks! let me see if I can find another solution. I think we found a misconfiguration in our Windows tests. We can now reproduce on GitHub
👍 1
I believe this https://github.com/PrefectHQ/prefect-shell/pull/70 should fix it. I just need to fix the Windows assertion tests If you’d like to help confirm:
pip install -U git+<https://github.com/PrefectHQ/prefect-shell.git@flush_after_write>
c

Carlos Cueto

02/17/2023, 8:38 PM
It works good now! Appreciate you taking the time to fix this.
🎉 1
a

Andrew Huang

02/17/2023, 8:38 PM
No problem! Thank you for raising it. I’ll try to get a release soon..
Released; should be up on pypi in a few minutes
👍 1
🙌 1
c

Carlos Cueto

02/17/2023, 10:34 PM
Thank you for the quick fix.
:thank-you: 2
a

Andrew Huang

02/17/2023, 10:35 PM
Thanks for reporting!
:upvote: 2