https://prefect.io logo
Join Slack
Powered by
Do we need to authenticate with prefect cloud via ...
# prefect-community
i
Do we need to authenticate with prefect cloud via 
prefect cloud login -k $PREFECT_API_KEY -w $PREFECT_WORKSPACE
in order to run a 
block.save
Python method? If we try authenticating through simply setting our default profile and running 
prefect config set PREFECT_API_KEY/PREFECT_API_URL
but not explicitly calling prefect cloud, we get an error when trying to save a storage block (noted in replies). Is there a recommended pattern to use for CI/CD to a save a block otherwise?
Copy code
Traceback (most recent call last):
  File "/home/runner/work/gha-prefect/gha-prefect/flows/flow_a/store_flow.py", line 25, in <module>
    storage_block = create_storage_block(bucket, flow)
  File "/home/runner/work/gha-prefect/gha-prefect/flows/flow_a/store_flow.py", line 16, in create_storage_block
    block.save(block_name, overwrite=True)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 193, in wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 140, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/prefect/blocks/core.py", line 724, in save
    document_id = await self._save(name=name, overwrite=overwrite)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/prefect/blocks/core.py", line 677, in _save
    await self.register_type_and_schema(client=client)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/prefect/blocks/core.py", line 619, in register_type_and_schema
    block_type = await client.read_block_type_by_slug(
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/prefect/client.py", line 1120, in read_block_type_by_slug
    return BlockType.parse_obj(response.json())
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/httpx/_models.py", line 743, in json
    return jsonlib.loads(self.text, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Error: Process completed with exit code 1.
k
Here is an example of how you can use Prefect for CI/CD. Can you try that to see if it works?
i
Hi Khuyen — yes, we’re doing the same steps from the Create blocks job here and receiving the above error and receiving the error above.
If we add prefect cloud login after line 53 this works but otherwise we get an error.
k
is there a reason why you don’t want to use prefect cloud login?
i
Based on Anna’s response here, it seems like we should be avoiding prefect cloud login for CI/CD? https://prefect-community.slack.com/archives/CL09KU1K7/p1661255009895719?thread_ts=1661254334.222679&amp;cid=CL09KU1K7
k
Do you get a prom for a profile name when using prefect cloud login?
i
We are running this in github actions so there’s no way to respond to a prompt so I’m not positive, but we are also running the command 
prefect profile use default
before 
prefect cloud login
to avoid the prompt.
I think otherwise the prompt will come up yes
k
I think either ways is fine (using prefect cloud login or prefect config set). When using prefect config set, you use the command like the following right?
Copy code
prefect profile use xyz
prefect config set PREFECT_API_KEY="xxx"
prefect config set PREFECT_API_KEY="yyy"
i
Exactly
Do we know what might be causing this error?
k
Could you test if you get the same error when running this in your local machine?
i
How do I de-authenticate with cloud CLI? Finding it difficult to test on my local since it’s been authenticated with cloud already and doesn’t seem to want to disconnect.
k
you can just delete the current profile
Copy code
prefect profile delete <name-of-your-profile>
Then run 
prefect cloud login -k API-KEY
✅ 1
i
Thanks Khuyen. I’ve tried now on my local as well and just creating a new profile, using 
prefect config set PREFECT_API_KEY/PREFECT_API_URL
and running the below script is generating the same error as described before.
Copy code
from prefect.filesystems import S3

def create_storage_block(bucket, flow):
    block_name = f"{flow}-code"

    aws_access_key_id = "xyz123"
    aws_secret_access_key = "xyz123"

    block = S3(
        bucket_path=f"{bucket}/flows/{flow}",
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key,
    )
    block.save(block_name, overwrite=True)

    return block_name


if __name__ == "__main__":
    bucket = "mybucket"
    flow = "myflow"

    storage_block = create_storage_block(bucket, flow)
And again if I authenticate with cloud using 
prefect cloud login
, the script works.
k
Can you show the output when running:
Copy code
prefect config view
Just to double check
i
Copy code
PREFECT_PROFILE='xyz2'
PREFECT_API_KEY='my_api_key' (from profile)
PREFECT_API_URL='<https://app.prefect.cloud/accounts/xyz/workspaces/xyz>' (from profile)
I do notice when I run 
prefect cloud login -k my_key
afterwards it’s switching profile back to default from xyz2
Copy code
prefect cloud login -k my_key                                                                                                          01:43:54 PM
Logged in to Prefect Cloud using profile 'default'.
Workspace is currently set to 'account/workspace'. The workspace can be changed using `prefect cloud workspace set`.
But in absence of doing prefect cloud login I am receiving the JSONDecodeError
t
I added a 
LocalFileSystem
block in the cloud but get the same `JSONDecodeError`when I try to use the block locally. Did you find a solution for this already?
i
Hi Tim — the only “workaround” I’ve found is to actually login to cloud CLI before trying to save our storage block.
3 Views
Open in Slack
PreviousNext
Powered by