Thread
#prefect-community
    m

    MasatoShima

    5 months ago
    Hi, I’m trying Prefect Orion. I am planning to use Amazon S3 bucket for Storage. I wrote the following code. However, I get an error. I have not created the Storage yet… Am I doing something wrong? I’d be glad to hear from you if anyone has any ideas!
    s3_storage_block = S3StorageBlock(
        bucket="********",
        profile_name="default",
        region_name="ap-northeast-1",
    )
    
    async with get_client() as client:
        block_id = await client.create_block(
            name="********",
            block=s3_storage_block,
            block_spec_id=uuid.UUID("{12345678-1234-5678-1234-567891234567}")
        )
    Traceback (most recent call last):
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/prefect/client.py", line 850, in create_block
        response = await <http://self._client.post|self._client.post>(
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/httpx/_client.py", line 1820, in post
        return await self.request(
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/httpx/_client.py", line 1506, in request
        return await self.send(request, auth=auth, follow_redirects=follow_redirects)
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 36, in send
        response.raise_for_status()
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/httpx/_models.py", line 1510, in raise_for_status
        raise HTTPStatusError(message, request=request, response=self)
    httpx.HTTPStatusError: Client error '409 Conflict' for url '<http://ephemeral-orion/api/blocks/>'
    For more information check: <https://httpstatuses.com/409>
    The above exception was the direct cause of the following exception:
    Traceback (most recent call last):
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3361, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
      File "<ipython-input-2-61bef0511626>", line 1, in <cell line: 1>
        runfile('/Users/********/PyCharmProjects/********/workflows/initialize.py', wdir='/Users/********/PyCharmProjects/********/workflows')
      File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
        pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
      File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
        exec(compile(contents+"\n", file, 'exec'), glob, loc)
      File "/Users/********/PyCharmProjects/********/workflows/initialize.py", line 218, in <module>
        initialize()
      File "/Users/********/PyCharmProjects/********/workflows/initialize.py", line 68, in initialize
        asyncio.run(create_storage())
      File "/Users/********/.pyenv/versions/3.9.1/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/Users/********/.pyenv/versions/3.9.1/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/Users/********/PyCharmProjects/********/workflows/initialize.py", line 128, in create_storage
        block_id = await client.create_block(
      File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/prefect/client.py", line 856, in create_block
        raise prefect.exceptions.ObjectAlreadyExists(http_exc=e) from e
    prefect.exceptions.ObjectAlreadyExists
    Anna Geller

    Anna Geller

    5 months ago
    Afaik, the storage blocks are not directly exposed yet via a Python client. To create storage, you can use the
    prefect storage create
    CLI command. Check this doc https://orion-docs.prefect.io/concepts/storage/
    m

    MasatoShima

    5 months ago
    Oh, I see. Thank you for your help.