Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
j
Jeremy Phelps
06/29/2021, 6:15 PMHi all,
I've lost the ability to retrieve logs using the CLI tool:
agent@prefect-agent:~$ prefect get logs --name granite-orca > log.log
Traceback (most recent call last):
File "/home/agent/.pyenv/versions/3.7.3/bin/prefect", line 10, in <module>
sys.exit(cli())
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/prefect/cli/get.py", line 430, in logs
result = Client().graphql(query)
File "/home/agent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/prefect/client/client.py", line 319, in graphql
raise ClientError(result["errors"])
prefect.utilities.exceptions.ClientError: [{'path': ['flow_run'], 'message': 'Operation timed out', 'extensions': {'code': 'API_ERROR'}}]k
Kevin Kho
06/29/2021, 6:25 PMHey @Jeremy Phelps, could you share the query you’re trying to run?
j
Jeremy Phelps
06/29/2021, 6:26 PMWhatever query
prefect get logs --name granite-orcak
Kevin Kho
06/29/2021, 6:26 PMOk will take a look
So there’s just too many logs for the query to retrieve with the API, the feedback is to use the Client and limit the logs being retrieved with something like
logs_query = {
with_args(
"logs",
{
"order_by": {EnumValue("timestamp"): EnumValue("asc")},
"where": {
"_and": [
{"timestamp": {"_lte": END_TIME}},
{"timestamp": {"_gt": START_TIME}},
]
},
},
): {"timestamp": True, "message": True, "level": True}
}
result = client.graphql(
{
"query": {
with_args(
"flow_run",
{
"where": {"id": {"_eq": FLOW_RUN_ID}},
},
): logs_query
}
}
)
# Unpack the result
logs = result.get("data", {}).get("flow_run", [{}])[0].get("logs", [])j
Jeremy Phelps
06/29/2021, 6:59 PMWhy can't the CLI tool do something smarter, like using that same query to fetch the logs in chunks?
I notice the query has "limit" and "offset" parameters, which are used by the Web UI.
k
Kevin Kho
06/29/2021, 7:01 PMWill ask the team about adding those options to the CLI and opening a ticket for it
@Marvin open “Add offset and limit to
prefect get logsm
Marvin
06/29/2021, 7:08 PM