Elliott Wilson
01/05/2023, 7:42 PMFlow could not be retrieved from deployment.
from boto3 when I try and run the deployment into the ec2 instance. I can deploy to s3 locally and connect to bucket from the ec2 using the AWS cli. Please can anyone help me debug this?Elliott Wilson
01/05/2023, 7:42 PMFlow could not be retrieved from deployment.
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 112, in _error_wrapper
return await func(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/aiobotocore/client.py", line 358, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/engine.py", line 262, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/deployments.py", line 166, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/filesystems.py", line 468, in get_directory
return await self.filesystem.get_directory(
File "/home/ubuntu/.local/lib/python3.10/site-packages/prefect/filesystems.py", line 313, in get_directory
return self.filesystem.get(from_path, local_path, recursive=True)
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 113, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 98, in sync
raise return_result
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
result[0] = await coro
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 561, in _get
return await _run_coros_in_chunks(
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 269, in _run_coros_in_chunks
await asyncio.gather(*chunk, return_exceptions=return_exceptions),
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 1134, in _get_file
body, content_length = await _open_file(range=0)
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 1125, in _open_file
resp = await self._call_s3(
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 339, in _call_s3
return await _error_wrapper(
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 139, in _error_wrapper
raise err
PermissionError: Access Denied
Zanie
Elliott Wilson
01/05/2023, 8:02 PMZanie
Elliott Wilson
01/05/2023, 8:02 PMZanie
Zanie
Zanie
Elliott Wilson
01/05/2023, 8:04 PMZanie
~/.aws/credentials
or in the IAM metadata or?Zanie
Elliott Wilson
01/05/2023, 8:06 PMElliott Wilson
01/05/2023, 8:07 PMAWS_ACCESS_KEY_ID
, AWS_SECRET_ACCESS_KEY
Elliott Wilson
01/05/2023, 8:07 PMZanie
Zanie
Elliott Wilson
01/05/2023, 8:08 PMZanie
env={"EXAMPLE_VARIABLE": None}
in the Process
infrastructure block.Zanie
Zanie
Elliott Wilson
01/05/2023, 8:10 PMElliott Wilson
01/05/2023, 8:11 PMZanie
Elliott Wilson
01/05/2023, 8:16 PMElliott Wilson
01/05/2023, 8:17 PMElliott Wilson
01/05/2023, 8:21 PMElliott Wilson
01/05/2023, 8:22 PMElliott Wilson
01/05/2023, 8:25 PMZanie
Elliott Wilson
01/05/2023, 8:54 PMZanie
Elliott Wilson
01/05/2023, 10:19 PMElliott Wilson
01/05/2023, 10:19 PMZanie
python -m prefect.engine <flow-run-id>
Zanie
Zanie
Zanie
Zanie
Elliott Wilson
01/06/2023, 1:23 PM