Choenden Kyirong
04/17/2023, 9:37 PMS3
to persist results of certain tasks due to them producing large amounts of data.
However, i’m running into an issue when trying to use: _`persist_result=True` and result_storage_=S3(bucket_path=<bucket_path>)
. I’m following along the documentation and doing the following:
...
from prefect.filesystems import S3
...
@task(name="<task_name>", persist_result=True,
result_storage_key="<key_name>")
def task_1(...):
...
@flow(task_runner=DaskTaskRunner, persist_result=True,
result_storage=S3(bucket_path="<bucket_path>"),
name="<name>")
def flow(...):
...
However, i’m running into this error when task_1
is running:
Crash detected! Execution was interrupted by an unexpected exception: botocore.exceptions.NoCredentialsError: Unable to locate credentials
I’m wondering if i’m using this correctly? I have registered the S3
block with the proper credentials and the documented example does not show a need to load
the block and just uses it like the above. It seems as though the call to S3
is unable to identify or locate any credentials. Any advice or help would be greatly appreciated!