It occurs between two tasks, where the first task is passing a list of filenames to the second. I suspect that list of filenames results in a query that is too large? But surely that is not sent to the API?

and you’ll probably want to look at the logs in STDOUT because there’s a chance this log will be rejected by the API as well because it might be large

Hi @Chris White, here is the output of the state handler:

State <Running: "Starting task run."> has serialized representation: {'message': 'Starting task run.', 'context': {'tags': []}, 'cached_inputs': {}, '_result': {'__version__': '0.13.18', 'type': 'NoResultType'}, '__version__': '0.13.18', 'type': 'Running'}

Although from looking at the stdout logs, it appears that my code is somehow logging contents of a file to stdout. As I am using

log_stdout=True

for this task, it seems thats the culprit. Very strange though that its outputting the contents of a file. This is one of the methods that gets called from within the task

def clean_file(filepath: str):
    print(f"Cleaning file {filepath}")
    filename = path.basename(filepath)
    with open(filepath, "r") as fin:
        data = fin.read().splitlines(True)
        for index, value in enumerate(data):
            data[index] = "|".join([filename, value])
    with open(filepath, "w") as fout:
        # skip line 1 (header) and skip last line (footer)
        fout.writelines(data[1 : len(data) - 1])
    return filepath

It appears

fout.writelines

is coming up on stdout rather than into the file. Or am I mistaken?

I’ve removed all

log_stdout

and explicity use the prefect logger, but I’m still seeing the actual file written to stdout. It used to work though, so very confused 😕

Locally when running

flow.run()

I don’t have this issue though

Hey @Chris White, good morning 🙂 Any ideas? Do you have any idea why writing to a file is logged to stdout when running in Kuebrnetes, but not locally?

It’s causing one of our main ETL jobs to fail :(

Thanks!