https://prefect.io logo
Title
w

Wenceslao Negrete

01/24/2023, 7:47 PM
Hi, I'm using Google Cloud Run Job block as Infra to run my flows. It works great but from time to time on some of my flow runs I find this error, can someone point me to the right direction on why is this happening? Details of the error on the thread
1
Submission failed. googleapiclient.errors.HttpError: <HttpError 429 when requesting <https://us-east1-run.googleapis.com/apis/run.googleapis.com/v1/namespaces/****/jobs/prefect-36a19b47578346329e4571058fef34b8:run?alt=json> returned "Unable to create EXECUTION "prefect-36a19b47578346329e4571058fef34b8-2hn86" because 6 already exist in this region.". Details: "Unable to create EXECUTION "prefect-36a19b47578346329e4571058fef34b8-2hn86" because 6 already exist in this region."
p

Peyton Runyan

01/24/2023, 8:46 PM
Interesting! How frequently are you getting them (every N runs) and are you doing anything fancy with your logic around running the deployment?
Is it always 6?
w

Wenceslao Negrete

01/24/2023, 9:08 PM
It varies from 6 to 8. Happens on flows that are scheduled to ran each 20 mins or when ad-hoc calls are called simultaneously even 2 or 3. Don't know if its related but the number of scheduled flows is 7.
Nothing fancy, grabbing data from a source to send it to another service based on a couple of conditions.
p

Peyton Runyan

01/25/2023, 3:19 PM
This seems related to run quotas for GCP cloud run jobs. Can you give this a look and check out your usage patterns and see if you see anything that overlaps? https://cloud.google.com/run/quotas
If you're doing it in smaller batches, I'm guessing it's one of the 10 limits, possibly the one under this subheading: https://cloud.google.com/run/quotas#api
🙌 1
w

Wenceslao Negrete

01/26/2023, 12:47 AM
Mhhh yeah, that's right. Makes total sense, thanks for the guidance @Peyton Runyan!