Pedro Machado

    Pedro Machado

    1 year ago
    Hi there. I have a flow that will run on Kubernetes. I'd like to see log messages in near real time. Currently, the messages show at the end of the run? I wonder if this has to do with the stream not being flushed frequently. How can I flush the stream after calling
    <http://logger.info|logger.info>()
    ? Thanks!
    nicholas

    nicholas

    1 year ago
    Hi @Pedro Machado - this isn't really possible because of how logs are batched; if your run is fairly short there's a good chance logs will always show up after the run has completed
    Pedro Machado

    Pedro Machado

    1 year ago
    Thanks, Nicholas. Would it be different in a long running flow? We'll have some of those and I want to make sure that we don't have to wait hours to see what's happening.
    nicholas

    nicholas

    1 year ago
    I would run a test if possible to confirm but since batching happens at some interval you should logs show up regularly, not just at the end of the run
    Pedro Machado

    Pedro Machado

    1 year ago
    Actually, the messages are showing in Prefect cloud as they are issued. The issue is with
    prefect run --project myproject -n "myflow" --watch
    They don't show in real time. Am I missing another option?
    nicholas

    nicholas

    1 year ago
    I don't think so, are you still seeing them only after the run has completed?
    Pedro Machado

    Pedro Machado

    1 year ago
    Correct. I see
    Task 'hello_task': Starting task run...
    And then it stays there until the flow finishes and at that time, it shows the remaining log entries.