Jeremy Phelps

    Jeremy Phelps

    1 year ago
    Hi all, I've lost the ability to retrieve logs using the CLI tool:
    agent@prefect-agent:~$ prefect get logs --name granite-orca > log.log
    Traceback (most recent call last):
      File "/home/agent/.pyenv/versions/3.7.3/bin/prefect", line 10, in <module>
        sys.exit(cli())
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/prefect/cli/get.py", line 430, in logs
        result = Client().graphql(query)
      File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/prefect/client/client.py", line 319, in graphql
        raise ClientError(result["errors"])
    prefect.utilities.exceptions.ClientError: [{'path': ['flow_run'], 'message': 'Operation timed out', 'extensions': {'code': 'API_ERROR'}}]
    I'm not sure what's going on, I had no trouble until recently. The Web UI can still retrieve logs, but it takes too long to scroll through them as it only keeps 100 entries in memory at a time.
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Jeremy Phelps, could you share the query you’re trying to run?
    Jeremy Phelps

    Jeremy Phelps

    1 year ago
    Whatever query
    prefect get logs --name granite-orca
    generates.
    Kevin Kho

    Kevin Kho

    1 year ago
    Ok will take a look
    So there’s just too many logs for the query to retrieve with the API, the feedback is to use the Client and limit the logs being retrieved with something like
    logs_query = {
                with_args(
                    "logs",
                    {
                        "order_by": {EnumValue("timestamp"): EnumValue("asc")},
                        "where": {
                            "_and": [
                                {"timestamp": {"_lte": END_TIME}},
                                {"timestamp": {"_gt": START_TIME}},
                            ]
                        },
                    },
                ): {"timestamp": True, "message": True, "level": True}
            }
            result = client.graphql(
                {
                    "query": {
                        with_args(
                            "flow_run",
                            {
                                "where": {"id": {"_eq": FLOW_RUN_ID}},
                            },
                        ): logs_query
                    }
                }
            )
            # Unpack the result
            logs = result.get("data", {}).get("flow_run", [{}])[0].get("logs", [])
    Jeremy Phelps

    Jeremy Phelps

    1 year ago
    Why can't the CLI tool do something smarter, like using that same query to fetch the logs in chunks?
    I notice the query has "limit" and "offset" parameters, which are used by the Web UI.
    Kevin Kho

    Kevin Kho

    1 year ago
    Will ask the team about adding those options to the CLI and opening a ticket for it
    @Marvin open “Add offset and limit to
    prefect get logs
    CLI command”
    Marvin

    Marvin

    1 year ago