Elliott Wilson
01/05/2023, 7:42 PMFlow could not be retrieved from deployment.
from boto3 when I try and run the deployment into the ec2 instance. I can deploy to s3 locally and connect to bucket from the ec2 using the AWS cli. Please can anyone help me debug this?Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 112, in _error_wrapper
return await func(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/aiobotocore/client.py", line 358, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/engine.py", line 262, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/deployments.py", line 166, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/filesystems.py", line 468, in get_directory
return await self.filesystem.get_directory(
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/filesystems.py", line 313, in get_directory
return self.filesystem.get(from_path, local_path, recursive=True)
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 113, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 98, in sync
raise return_result
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
result[0] = await coro
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 561, in _get
return await _run_coros_in_chunks(
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 269, in _run_coros_in_chunks
await asyncio.gather(*chunk, return_exceptions=return_exceptions),
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 1134, in _get_file
body, content_length = await _open_file(range=0)
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 1125, in _open_file
resp = await self._call_s3(
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 339, in _call_s3
return await _error_wrapper(
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 139, in _error_wrapper
raise err
PermissionError: Access Denied
Zanie
01/05/2023, 8:02 PMElliott Wilson
01/05/2023, 8:02 PMZanie
01/05/2023, 8:02 PMElliott Wilson
01/05/2023, 8:02 PMZanie
01/05/2023, 8:03 PMElliott Wilson
01/05/2023, 8:04 PMZanie
01/05/2023, 8:06 PM~/.aws/credentials
or in the IAM metadata or?Elliott Wilson
01/05/2023, 8:06 PMAWS_ACCESS_KEY_ID
, AWS_SECRET_ACCESS_KEY
Zanie
01/05/2023, 8:08 PMElliott Wilson
01/05/2023, 8:08 PMZanie
01/05/2023, 8:08 PMenv={"EXAMPLE_VARIABLE": None}
in the Process
infrastructure block.Elliott Wilson
01/05/2023, 8:10 PMZanie
01/05/2023, 8:15 PMElliott Wilson
01/05/2023, 8:16 PMZanie
01/05/2023, 8:33 PMElliott Wilson
01/05/2023, 8:54 PMZanie
01/05/2023, 8:56 PMElliott Wilson
01/05/2023, 10:19 PMZanie
01/05/2023, 10:34 PMpython -m prefect.engine <flow-run-id>
Elliott Wilson
01/06/2023, 1:23 PM