Young Ho Shin
09/05/2022, 11:47 AMAnna Geller
09/05/2022, 12:51 PMYoung Ho Shin
09/06/2022, 5:36 AMAnna Geller
09/06/2022, 8:32 AMYoung Ho Shin
09/06/2022, 10:25 AMscript.py
is meant to be a place holder for CLI tools written in C/java/etc that cannot be easily written in python. For example, in my current processing pipeline, I am processing PCAP network capture files, which can be processed in python, but there are existing tools that do the same job much faster.
In any case, I think I've fixed a problem with my toy example. Using print("xyz", flush=True)
forced my python script to write to stdout every second, which is then handled correctly by`anyio`.
Now I get output like this, which is what I wanted:
19:17:06.027 | INFO | Task run 'shell_run_command-7398b6ba-3' - Been running task for 0 seconds... Repeating message: meow
19:17:06.257 | INFO | Task run 'shell_run_command-7398b6ba-2' - Been running task for 0 seconds... Repeating message: quack
19:17:06.447 | INFO | Task run 'shell_run_command-7398b6ba-0' - Been running task for 0 seconds... Repeating message: moo
19:17:07.028 | INFO | Task run 'shell_run_command-7398b6ba-3' - Been running task for 1 seconds... Repeating message: meow
19:17:07.067 | INFO | Task run 'shell_run_command-7398b6ba-1' - Been running task for 0 seconds... Repeating message: woof
19:17:07.258 | INFO | Task run 'shell_run_command-7398b6ba-2' - Been running task for 1 seconds... Repeating message: quack
DaskTaskRunner
instead of the default task runner, but maybe that's related to open issues currently being worked on:
https://github.com/PrefectHQ/prefect/issues/6022
https://github.com/PrefectHQ/prefect/issues/5850Anna Geller
09/10/2022, 12:01 AM