https://prefect.io logo
Channels
Powered by
n

Nelson Griffiths

05/11/2023, 3:00 PM
I am running into some issues trying to get prefect projects working with 
prefect-gcp
credentials blocks. When I try to pass my credentials in via a block I get 
AttributeError: 'str' object has no attribute 'keys'
Ill share the full traceback in the thread. But it seems like it is having a hard time converting my credentials block into actual credentials.
Copy code
Traceback (most recent call last):
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 260, in coroutine_wrapper
    return call()
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 245, in __call__
    return self.result()
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
    return self.future.result(timeout=timeout)
  File "/Users/nelsong/.pyenv/versions/3.10.8/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/Users/nelsong/.pyenv/versions/3.10.8/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
    result = await coro
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/cli/deploy.py", line 201, in deploy
    await _run_multi_deploy(
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/cli/deploy.py", line 526, in _run_multi_deploy
    await _run_single_deploy(base_deploy, project)
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/cli/deploy.py", line 394, in _run_single_deploy
    step_outputs.update(await run_step(step))
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/projects/steps/core.py", line 78, in run_step
    return await from_async.call_soon_in_new_thread(
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 181, in aresult
    return await asyncio.wrap_future(self.future)
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 194, in _run_sync
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/prefect_gcp/projects/steps.py", line 94, in push_project_to_gcs
    gcp_creds = Credentials.from_service_account_info(
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/google/oauth2/service_account.py", line 224, in from_service_account_info
    signer = _service_account_info.from_dict(
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/google/auth/_service_account_info.py", line 49, in from_dict
    missing = keys_needed.difference(six.iterkeys(data))
  File "/Users/nelsong/doubleriver/repos/data-monitoring/.venv/lib/python3.10/site-packages/six.py", line 599, in iterkeys
    return iter(d.keys(**kw))
AttributeError: 'str' object has no attribute 'keys'
n

Nate

05/11/2023, 3:07 PM
hi @Nelson Griffiths can you share how you're trying to template in your credentials?
n

Nelson Griffiths

05/11/2023, 3:51 PM
@Nate Here it is. I think I followed the example exactly?
Copy code
push:
- prefect_gcp.projects.steps.push_project_to_gcs:
    requires: prefect-gcp>=0.4.0
    bucket: dr-dev-us-east1-prefect-flows
    folder: data-monitoring
    credentials: '{{ prefect.blocks.gcp-credentials.gcp-creds }}'
n

Nate

05/11/2023, 3:55 PM
ah, can you try with double quotes? i think you're getting the literal string value by using single quotes
n

Nelson Griffiths

05/11/2023, 4:00 PM
I get the exact same error with double quotes
n

Nate

05/11/2023, 4:01 PM
hmm ok i will try to reproduce
did you create the secret through the UI or the python sdk? i think the secret value may be returned with the wrong type by the API here
n

Nelson Griffiths

05/11/2023, 4:58 PM
I created the block through the UI. Quite awhile ago
Should I try to create a new block and do it again?
a

alex

05/11/2023, 5:27 PM
I recommend recreating the block. There was an issue a while back with the GCP Credentials block where the service account info was saved as a JSON string instead of as an object. We can add handling to attempt to parse JSON strings so that this error doesn’t happen, but hopefully, recreating the block unblocks you.
👍 1
n

Nelson Griffiths

05/11/2023, 5:40 PM
@alex That worked thank you! Should've tried that first 😂
🎉 2
w

Wolfgang Steitz

05/11/2023, 6:43 PM
I created an issue yesterday with the same problem. https://github.com/PrefectHQ/prefect-gcp/issues/179. I'll try recreating the block
That worked. Thanks for the tip!
Should I leave the ticket open?
a

alex

05/12/2023, 4:14 PM
Yes you can leave the ticket open. We’ll add some type-checking and better error handling to prevent users from hitting this in the future.
2 Views
Powered by