Hi I am working with `S3Result` and receiving a ```botocore Prefect Community #ask-community

Hi - I am working with `S3Result` and receiving a...

itay livni

05/16/2020, 9:19 PM

Hi - I am working with

S3Result

and receiving a

Copy code

botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist

Which upon further research - it can be anything including a permission error. (I tried different buckets with settings) The credentials are stored as AWS_CREDENTIALS in prefect cloud. With the config.toml set to use cloud secrets

Copy code

[cloud]
use_local_secrets = false

Switching back to

result_handler

argument with

S3Result

subclass did work, . And combining

result handler

with

target

does not. Is there something different in the way that credentials are handled between

result

and

result_handler

? The new prefect is really nice 🙂

😄 2

Chris White

05/16/2020, 9:24 PM

Hi itay - could you share the code you used to initialize the

result_handler

and the

result

itay livni

05/16/2020, 9:24 PM

tsx_imb_res = S3Result(bucket="tsx-moc-bcp")

itay livni

05/16/2020, 9:25 PM

Copy code

@task(
    max_retries=3,
    retry_delay=timedelta(seconds=1), # In production this will be change to 20 minutes
    result_handler=tsx_imb_res, 
    target="{task_name}-{today}",
    state_handlers=[imb_handler, error_handler]
)

itay livni

05/16/2020, 9:25 PM

Works with

target

commented out

Chris White

05/16/2020, 9:26 PM

Ah! The

result_handler

kwarg is now deprecated, so you should instead try:

Copy code

...
result=tsx_imb_res,
...

itay livni

05/16/2020, 9:26 PM

Right that does not work

itay livni

05/16/2020, 9:26 PM

That is the problem

Chris White

05/16/2020, 9:28 PM

ahhh interesting! OK so this might actually be a bug with our

exists

logic on the

S3Result

type. Would you mind sharing this example code + the traceback you’re seeing? Sorry about that!

itay livni

05/16/2020, 9:32 PM

Actually it looks like I am not using

target

right?

Copy code

[2020-05-16 21:31:37] INFO - prefect.FlowRunner | Beginning Flow run for 'Our first flow'
[2020-05-16 21:31:37] INFO - prefect.FlowRunner | Starting flow run.
[2020-05-16 21:31:37] INFO - prefect.TaskRunner | Task 'tsx_url': Starting task run...
[2020-05-16 21:31:37] INFO - prefect.TaskRunner | Task 'tsx_url': finished task run for task with final state: 'Success'
[2020-05-16 21:31:37] INFO - prefect.TaskRunner | Task 'get_tsx_moc_imb': Starting task run...
[2020-05-16 21:31:38] ERROR - prefect.TaskRunner | Unexpected error: NoSuchKey('An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.')
Traceback (most recent call last):
  File "/home/ilivni/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
    new_state = method(self, state, *args, **kwargs)
  File "/home/ilivni/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 651, in check_target
    if result.exists(target, **prefect.context):
  File "/home/ilivni/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/results/s3_result.py", line 167, in exists
    Bucket=self.bucket, Key=location.format(**kwargs)
  File "/home/ilivni/miniconda3/envs/py37moc/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/ilivni/miniconda3/envs/py37moc/lib/python3.7/site-packages/botocore/client.py", line 626, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
[2020-05-16 21:31:38] INFO - prefect.TaskRunner | Task 'get_tsx_moc_imb': finished task run for task with final state: 'Skipped'
[2020-05-16 21:31:38] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded

itay livni

05/16/2020, 9:34 PM