https://prefect.io logo
#prefect-community
Title
# prefect-community
m

MasatoShima

04/17/2022, 7:20 AM
Hi, I’m trying Prefect Orion. I am planning to use Amazon S3 bucket for Storage. I wrote the following code. However, I get an error. I have not created the Storage yet… Am I doing something wrong? I’d be glad to hear from you if anyone has any ideas!
Copy code
s3_storage_block = S3StorageBlock(
    bucket="********",
    profile_name="default",
    region_name="ap-northeast-1",
)

async with get_client() as client:
    block_id = await client.create_block(
        name="********",
        block=s3_storage_block,
        block_spec_id=uuid.UUID("{12345678-1234-5678-1234-567891234567}")
    )
Copy code
Traceback (most recent call last):
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/prefect/client.py", line 850, in create_block
    response = await <http://self._client.post|self._client.post>(
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/httpx/_client.py", line 1820, in post
    return await self.request(
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/httpx/_client.py", line 1506, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 36, in send
    response.raise_for_status()
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/httpx/_models.py", line 1510, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '409 Conflict' for url '<http://ephemeral-orion/api/blocks/>'
For more information check: <https://httpstatuses.com/409>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3361, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-61bef0511626>", line 1, in <cell line: 1>
    runfile('/Users/********/PyCharmProjects/********/workflows/initialize.py', wdir='/Users/********/PyCharmProjects/********/workflows')
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/********/PyCharmProjects/********/workflows/initialize.py", line 218, in <module>
    initialize()
  File "/Users/********/PyCharmProjects/********/workflows/initialize.py", line 68, in initialize
    asyncio.run(create_storage())
  File "/Users/********/.pyenv/versions/3.9.1/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/********/.pyenv/versions/3.9.1/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/********/PyCharmProjects/********/workflows/initialize.py", line 128, in create_storage
    block_id = await client.create_block(
  File "/Users/********/Library/Caches/pypoetry/virtualenvs/********-z0nXSFMl-py3.9/lib/python3.9/site-packages/prefect/client.py", line 856, in create_block
    raise prefect.exceptions.ObjectAlreadyExists(http_exc=e) from e
prefect.exceptions.ObjectAlreadyExists
a

Anna Geller

04/17/2022, 10:15 AM
Afaik, the storage blocks are not directly exposed yet via a Python client. To create storage, you can use the
prefect storage create
CLI command. Check this doc https://orion-docs.prefect.io/concepts/storage/
🙏 1
m

MasatoShima

04/18/2022, 12:58 PM
Oh, I see. Thank you for your help.
9 Views