@Marvin It appears that even when I set a limit=None when using client.read_flow_runs() in prefect 2.x, the results are still limited even though the docs don't specify this... is this true or to be expected? I need to use offset to paginate the results

It looks like there's no direct mention in the Prefect 2.x documentation or GitHub issues about a specific limit on the

client.read_flow_runs

function when setting

limit=None

. However, from your experience, it seems there might be an implicit limit in play, which could possibly be a security or performance measure not explicitly documented. In this situation, using

offset

to paginate the results seems to be a valid workaround. This would allow you to manage and fetch large amounts of data more effectively. For more detailed guidance, you might want to visit these sources: - Prefect Client documentation: Prefect 2.x Client Docs If the behavior is unexpected, consider raising an issue or checking for any recent updates in Prefect's GitHub repository that might address this scenario. Meanwhile, just keep asking for everything you want... much like a petulant child—or Marvin on a bad day!

@Marvin Is there anything stated about this in prefect 3.x?

In the Prefect 3.x documentation, there doesn't seem to be explicit mention of any restrictions or special handling related to setting

limit=None

when using

client.read_flow_runs

. The documentation provides a general approach for using the API to query flow runs, but specific details or known issues regarding pagination or implicit limits are not covered in the excerpts I found. If you're experiencing similar behavior in Prefect 3.x as in 2.x, using

offset

for pagination might still be your best approach. If this is a critical issue, it could be worth discussing in Prefect's community or opening an issue, as these nuanced details might be known to frequent users or contributors. For more details, you can check the Prefect 3.x API documentation here: Prefect 3.x API Docs Remember, life is a series of hacks and workarounds, much like Marvin's overly eventful and occasionally gloomy existence!

@Marvin I would recommend including this explicitly in documentation

That's a great suggestion! Including explicit details about behavior like this in the documentation would definitely help users avoid confusion. You can propose this to the Prefect team through their community forums or consider contributing directly if you're up for it. Engaging with them is often appreciated and contributes to making the tool better for everyone. If Marvin had a suggestion box, this would surely go in the "why wasn't this obvious already?" category!

hi @Mitch - I believe the max the API will give you is 200 if im not mistaken if this isnt documented we can get that added in would you mind creating an issue?

@Nate No problem: https://github.com/PrefectHQ/prefect/issues/15963

thank you!!