https://prefect.io logo
Title
h

Henning Holgersen

01/23/2023, 12:46 PM
Hi, I’m rewriting some prefect 1 flows to prefect 2, and I want to double-check one behaviour change: It seems that in prefect 1, all output from the shelltask command were returned, but in prefect-shell, it appears that logs (typically python logging libraries, output that isn’t technically printed to stdout but none the less shows up on the command line) don’t show up in the prefect logs. Am I understanding this correctly? Is there any way to catch these logs?
I tried setting stream_level=logging.DEBUG, but it does not seem to help. The logs I’m expecting should be at the INFO level anyways. I have tried to check that the task actually runs, and from what I can tell, it does.
k

Kirill Ulich

01/23/2023, 2:14 PM
Hi! What command are you running?
h

Henning Holgersen

01/23/2023, 2:15 PM
I’m running Meltano (
meltano run tap-something target-something
). I just now (a few seconds ago) saw the issue in the repo, so I am trying to run it through the prefect module.
k

Kirill Ulich

01/23/2023, 2:22 PM
You can try adding StreamHandler(sys.stdout) to the meltano logger
h

Henning Holgersen

01/23/2023, 2:31 PM
Thanks for the suggestion. I’m not very experienced with logging but I can imagine what it does. Sadly, the logging messages comes from code I don’t maintain, and Meltano (a wrapper for singer.io connectors) uses stdout for the application itself. Adding the logging messages to stdout would in all likelihood make the application fail. This isn’t really critical though, it would just be nice to have.
k

Kirill Ulich

01/23/2023, 2:39 PM
You can to change logger configuration https://docs.meltano.com/guide/logging
👀 1
:thank-you: 1