Andrew Lawlor
04/21/2022, 6:17 PMFailed to load and execute flow run: RefreshError(TransportError("Failed to retrieve <http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/ACCOUNT/token?scopes=SCOPES> from the Google Compute Engine metadata service. Status: 500 Response:\nb'Unable to generate access token; IAM returned \\n'", <google.auth.transport.requests._Response object at 0x7f5b071fd670>))
Anna Geller
04/21/2022, 6:20 PMroles/iam.workloadIdentityUser
.
Re-issuing gcloud iam service-account add-iam-policy-binding
command fixed the issue."Andrew Lawlor
04/21/2022, 6:31 PMAnna Geller
04/21/2022, 6:36 PMAndrew Lawlor
04/21/2022, 6:39 PMrun_config = KubernetesRun(
image=docker_image,
labels=[project_id],
cpu_request='200m',
memory_request='128Mi',
env={ENV_VARS}
)
Anna Geller
04/21/2022, 6:41 PMAndrew Lawlor
04/21/2022, 6:56 PMAnna Geller
04/21/2022, 7:18 PMAndrew Lawlor
04/21/2022, 7:20 PMAnna Geller
04/21/2022, 7:22 PMAndrew Lawlor
04/21/2022, 7:23 PMgoogle.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <https://cloud.google.com/docs/authentication/getting-started>
again, it doesnt happen everytime, only if lots of flows are running at once. my credenttials are set on pods, so i think its also related to a rate limit. this time it happens on connecting to bigqueryAnna Geller
04/21/2022, 8:09 PMAndrew Lawlor
04/21/2022, 8:12 PMAnna Geller
04/21/2022, 8:16 PMAndrew Lawlor
04/21/2022, 8:18 PM