Will Milner

    Will Milner

    1 year ago
    is there any extra config needed in order to have std_out from a ShellTask show up in the server logs? When I declare a task like this I see the output fine
    @task(log_stdout=True)
    def sample_print_task():
        print("hello")
    When I declare a shell task like this
    task_shell = ShellTask(return_all=True, log_stdout=True, log_stderr=True, stream_output=True)
    with Flow("test") as flow:
        print_test = task_shell(command="echo hi", task_args={"name": "hi"})
    I don't see anything printed after I register and run the flow. I have
    log_to_cloud
    set to True on the agent I am running
    j

    josh

    1 year ago
    Ah that would make for a good enhancement if you want to open an issue for it on the repo! 🙂 Currently the shell task looks to not do anything with the base task class’s
    log_stdout
    so you would need to do something like:
    @task(log_stdout=True)
    def show_output(lines):
        print(lines)
    
    # OR
    
    @task
    def show_output(lines):
        from prefect import context
        logger = context.get("logger")
        <http://logger.info|logger.info>(lines)
    
    task_shell = ShellTask(return_all=True, log_stderr=True, stream_output=True)
    with Flow("test") as flow:
        print_test = task_shell(command="echo hi", task_args={"name": "hi"})
        show_output(print_test)
    Will Milner

    Will Milner

    1 year ago
    that works, but only if the output is small enough. One of my tasks returned an error with
    Failed to write log with error: 413 Client Error: Payload Too Large for url: <http://host.docker.internal:4200/graphql>
    looking at the implementation of ShellTask it should be enough to just have stream_output=True, but I'm still not seeing any logs from any ShellTasks appear
    j

    josh

    1 year ago
    That could be due to debug logs not being on by default. For your agent try setting the env var
    --env PREFECT__LOGGING__LEVEL=DEBUG
    (this will pass the env var to all flow runs it creates) or set it on the single flow’s run config directly
    Will Milner

    Will Milner

    1 year ago
    I have both
    PREFECT__LOGGING__LEVEL=DEBUG
    and
    PREFECT__LOGGING_LOG_TO_CLOUD=True
    on my agent, still no logs from ShellTasks
    j

    josh

    1 year ago
    Interesting 🤔 could you open an issue with how you are running everything / a reproducible example on the repo because that sounds like it could be a bug
    Will Milner

    Will Milner

    1 year ago
    ok so when in my efforts to create a reproducible example, everything was actually working fine. Turns out my issue was in my docker image there was a different version of prefect installed vs. what my server was running on
    sorry for the confusion and thanks for the help