https://prefect.io logo
w

Will Milner

12/16/2020, 2:30 PM
is there any extra config needed in order to have std_out from a ShellTask show up in the server logs? When I declare a task like this I see the output fine
Copy code
@task(log_stdout=True)
def sample_print_task():
    print("hello")
When I declare a shell task like this
Copy code
task_shell = ShellTask(return_all=True, log_stdout=True, log_stderr=True, stream_output=True)
with Flow("test") as flow:
    print_test = task_shell(command="echo hi", task_args={"name": "hi"})
I don't see anything printed after I register and run the flow. I have
log_to_cloud
set to True on the agent I am running
j

josh

12/16/2020, 2:33 PM
Ah that would make for a good enhancement if you want to open an issue for it on the repo! 🙂 Currently the shell task looks to not do anything with the base task class’s
log_stdout
so you would need to do something like:
Copy code
@task(log_stdout=True)
def show_output(lines):
    print(lines)

# OR

@task
def show_output(lines):
    from prefect import context
    logger = context.get("logger")
    <http://logger.info|logger.info>(lines)

task_shell = ShellTask(return_all=True, log_stderr=True, stream_output=True)
with Flow("test") as flow:
    print_test = task_shell(command="echo hi", task_args={"name": "hi"})
    show_output(print_test)
w

Will Milner

12/16/2020, 2:39 PM
that works, but only if the output is small enough. One of my tasks returned an error with
Failed to write log with error: 413 Client Error: Payload Too Large for url: <http://host.docker.internal:4200/graphql>
looking at the implementation of ShellTask it should be enough to just have stream_output=True, but I'm still not seeing any logs from any ShellTasks appear
j

josh

12/16/2020, 2:43 PM
That could be due to debug logs not being on by default. For your agent try setting the env var
Copy code
--env PREFECT__LOGGING__LEVEL=DEBUG
(this will pass the env var to all flow runs it creates) or set it on the single flow’s run config directly
w

Will Milner

12/16/2020, 2:45 PM
I have both
PREFECT__LOGGING__LEVEL=DEBUG
and
PREFECT__LOGGING_LOG_TO_CLOUD=True
on my agent, still no logs from ShellTasks
j

josh

12/16/2020, 2:49 PM
Interesting 🤔 could you open an issue with how you are running everything / a reproducible example on the repo because that sounds like it could be a bug
w

Will Milner

12/16/2020, 3:16 PM
ok so when in my efforts to create a reproducible example, everything was actually working fine. Turns out my issue was in my docker image there was a different version of prefect installed vs. what my server was running on
sorry for the confusion and thanks for the help
2 Views