Hi- I was wondering if there was a prefect cloud interruption today. My flows exited with 404 and Protocol Errors.
Copy code
Failed to send heartbeat with exception: HTTPError('404 Client Error: Not Found for url: <https://api.prefect.io/graphql>')
Traceback (most recent call last):
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/cli/heartbeat.py", line 68, in flow_run
client.update_flow_run_heartbeat(id)
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 1610, in update_flow_run_heartbeat
self.graphql(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 548, in graphql
result = <http://self.post|self.post>(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 451, in post
response = self._request(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 737, in _request
response = self._send_request(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 647, in _send_request
response.raise_for_status()
File "/opt/conda/envs/dev/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://api.prefect.io/graphql>
k
Kevin Kho
10/08/2021, 6:28 PM
There was but not today. More like 5:22 PM ET yesterday. Is there a time for this error? The outage lasted a few seconds
v
Vincent
10/08/2021, 6:30 PM
Gotcha. Yes It was around 5:25.
Vincent
10/08/2021, 6:32 PM
In the case of these communication errors - What happens to a flow?
k
Kevin Kho
10/08/2021, 6:36 PM
It retries like 3 times, and if it can’t establish communication, it dies. This specifically was a certificate issue and we updated it so it’s not expected to happen again. Normally if an API call fails it will just retry and succeed
Kevin Kho
10/08/2021, 6:36 PM
The rate limiting though is a different story. That one might continue to fail
v
Vincent
10/08/2021, 6:37 PM
I guess I am having really bad luck with timing my flows then. 😞
k
Kevin Kho
10/08/2021, 6:38 PM
I’ll ask about rate limiting for you
Kevin Kho
10/08/2021, 6:38 PM
The 404 is really just a one time thing at that time yesterday. Shouldnt be expected moving forward
v
Vincent
10/08/2021, 6:47 PM
Thanks for the update Kevin.
k
Kevin Kho
10/08/2021, 8:18 PM
Confirmed that for the last 24 hours your IP was blocked for 1 minute several times causing the 502. Are your flows bigger than before?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.