https://prefect.io logo
e

emre

03/19/2020, 10:02 AM
Hello Everyone I’ve had a mild issue about
ShellTask
, in which a
git clone
command doesn’t return some of the stdout lines (despite using
return_all=True
). The command is as follows:
git clone https://{some_url}.git dbt_project -v -b master --depth 1
. Running it from bash logs the following to stdout:
Copy code
Cloning into 'dbt_project'...
POST git-upload-pack (156 bytes)
POST git-upload-pack (165 bytes)
remote: Counting objects: 40, done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 40 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (40/40), done.
While running the same command through
ShellTask
, and iterating over the resulting list of stdout lines logs the following:
Copy code
[2020-03-19 09:57:50,792] INFO - prefect.Task: print_task | Cloning into 'dbt_project'...
[2020-03-19 09:57:50,792] INFO - prefect.Task: print_task | POST git-upload-pack (168 bytes)
[2020-03-19 09:57:50,793] INFO - prefect.Task: print_task | POST git-upload-pack (177 bytes)
By any means, I am not well versed on bash, and this may very well be an issue about how
git clone
logs to stdout. Wondering if anybody else encountered a similar issue, and if they managed to come up with a fix?
t

trapped

03/19/2020, 10:53 AM
I guess it's possible that git is printing that missing output only if it detects an interactive session, or that the
ShellTask
is not picking up stderr altogether from the subprocess
e

emre

03/19/2020, 1:07 PM
Thanks, git was indeed printing the progress status to stderr, which isn’t picked up by
ShellTask
. The
--progress
flag forces git to print progress logs to stdout, as far as I understand.