https://prefect.io logo
#prefect-community
Title
# prefect-community
h

Howard Cornwell

06/09/2020, 3:15 PM
Is there a set (or estimated) limit for the amount of data you can cache with a
PrefectResult
? I’ve hit a wall and flows are failing with:
Copy code
Failed to set task state with error: HTTPError('413 Client Error: Payload Too Large for url: http://???:4200/graphql/graphql/alpha')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/engine/cloud/task_runner.py", line 119, in call_runner_target_handlers
    state = self.client.set_task_run_state(
  File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 1096, in set_task_run_state
    result = self.graphql(
  File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 213, in graphql
    result = <http://self.post|self.post>(
  File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 172, in post
    response = self._request(
  File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 318, in _request
    response.raise_for_status()
  File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 413 Client Error: Payload Too Large for url: http://???:4200/graphql/graphql/alpha
z

Zachary Hughes

06/09/2020, 3:19 PM
Hi @Howard Cornwell! Are you using Cloud or server for orchestration? It looks like Server, but don't want to make assumptions.
h

Howard Cornwell

06/09/2020, 3:19 PM
I’m using server
z

Zachary Hughes

06/09/2020, 3:23 PM
Okay, thanks! Do you mind creating an issue for this? We don't actively enforce a payload limit here, so this is unintentional.
h

Howard Cornwell

06/09/2020, 3:26 PM
Oh interesting. Sure, I’ll raise an issue. I’ll check what size the data being serialized is and include that too
I didn’t think it was that large, but I’ll make sure
Thanks for the prompt response
z

Zachary Hughes

06/09/2020, 3:27 PM
It would be wonderful if you could find the file size. Let me know if there's any way I can help you dig there!
h

Howard Cornwell

06/09/2020, 3:29 PM
I think (i hope?!) I know what’s being cached so I should be able to dump it to disk locally
🎉 2
z

Zachary Hughes

06/09/2020, 4:06 PM
You rock-- thank you!
✔️ 1
h

Howard Cornwell

06/11/2020, 9:56 AM
Hey, just tried the fix you applied and it’s working as expected. Now I just need to find 1pb of data to throw at it
5 Views