Hello, I am running an Azure DevOps pipeline with service connection set up to access my AWS account. As a pipeline task, I run a sagemaker processing job that includes a Prefect flow.

- task: AWSShellScript@1
              inputs:
                # awsCredentials: 'xxxxx'
                regionName: 'xxx'
                scriptType: 'inline'
                inlineScript: python3 cli.py -remote S3 -p "['a','2020-09-01', '2020-09-02']"

I get this error:

[2022-02-03 21:29:07+0000] ERROR - prefect.S3Result | Unexpected error while reading from S3: TypeError('expected string or bytes-like object')
Traceback (most recent call last):
  File "/miniconda3/lib/python3.7/site-packages/prefect/engine/results/s3_result.py", line 142, in exists
    self.client.get_object(Bucket=self.bucket, Key=location.format(**kwargs))
  File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 391, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 692, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 738, in _convert_to_request_dict
    api_params, operation_model, context)
  File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 770, in _emit_api_params
    params=api_params, model=operation_model, context=context)
  File "/miniconda3/lib/python3.7/site-packages/botocore/hooks.py", line 357, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/miniconda3/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/miniconda3/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/miniconda3/lib/python3.7/site-packages/botocore/handlers.py", line 238, in validate_bucket_name
    if not VALID_BUCKET.search(bucket) and not VALID_S3_ARN.search(bucket):
TypeError: expected string or bytes-like object

the same task runs good when I just run it on an Azure Devops agent:

- task: AWSShellScript@1
              inputs:
                # awsCredentials: 'xxxxx'
                regionName: 'xxx'
                scriptType: 'inline'
                inlineScript: python3 cli.py -local S3 -p "['a','2020-09-01', '2020-09-02']"

has anyone had this issue before?

Can I see your S3 result definition?

Never mind. I found the issue. ADO pipeline variables were not passed to my prefect flow. I had to supplement them as a separated pipeline step

Ah I see thanks for letting me know 🙂