Are there limits on how many STDOUT lines will be captured in a given timeframe?
When using
print()
statements in code and using the
@task(log_stdout=True)
decorator, running a particular flow manually emits 204 log lines, whereas only 103 lines end up in prefect.
jack
02/28/2022, 8:04 PM
In the lower right of this screenshot is a link "Load older logs"
k
Kevin Kho
02/28/2022, 8:11 PM
You’re on Cloud right? Is there a pattern to which lines end up like are they the older or later logs?
j
jack
02/28/2022, 8:13 PM
Yes, on cloud.
jack
02/28/2022, 8:15 PM
My guess is only the newest 100 log lines show up unless you click "Load older logs"
I'm going to move toward atomic logging, which will significantly reduce how many individual lines are logged.
k
Kevin Kho
02/28/2022, 8:15 PM
Ahh yes logs are paginated in the UI
j
jack
02/28/2022, 8:16 PM
Is there a flag I can add to the URL that will unpaginate them?
k
Kevin Kho
02/28/2022, 8:18 PM
I believe no. The GraphQL API call is the one the limits I think and it just has to do with performance
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.