Hello everyone I have a question about Orion s3 storage When

Question

Hello everyone I have a question about Orion s3 storage When I created a test deployment a file was created on my bucket but when I tried to run the deployment I got this error in the thread Of note I use corporate AWS account and do not have full S3 Access could this be the core of the issue

Anna Geller · Answer

< Azer Rustamov> can you move the traceback to the thread to keep the main channel clean

Anna Geller · Answer

1 Problem You get this error because you use `DockerFlowRunner` and didn t pass AWS credentials to your flow run container Currently your Docker container doesn t have the AWS credentials to pull the flow definition from S3 2 Solution To fix that you can mount your AWS credentials to your `DockerFlowRunner` as follows ```from prefect import flow get run logger from prefect deployments import DeploymentSpec from prefect flow runners import DockerFlowRunner flow def docker flow logger = get run logger <http logger info|logger info> Hello from Docker DeploymentSpec name= example flow=docker flow flow runner=DockerFlowRunner image= prefecthq prefect 2 0a13 python3 9 volumes= Users anna aws root aws ``` Note that you need to mount credentials to Docker root user otherwise it won t work

Azer Rustamov · Answer

Traceback ```Agent started 11 44 44 511 | INFO | prefect agent Submitting flow run ce7b588b 23ac 4757 8767 f53f412a4e6f 11 44 44 618 | INFO | prefect flow runner docker Flow run celadon cobra has container settings = image prefecthq prefect 2 0a13 python3 8 netw ork None command python m prefect engine ce7b588b 23ac 4757 8767 f53f412a4e6f environment PREFECT API URL <http host docker int> ernal 4200 api auto remove False labels io prefect flow run id ce7b588b 23ac 4757 8767 f53f412a4e6f extra hosts host docker internal host gateway name celadon cobra volumes 11 44 45 085 | INFO | prefect agent Completed submission of flow run ce7b588b 23ac 4757 8767 f53f412a4e6f 11 44 45 095 | INFO | prefect flow runner docker Flow run container celadon cobra has status running 11 44 49 756 | ERROR | prefect engine Engine execution of flow run ce7b588b 23ac 4757 8767 f53f412a4e6f exited with unexpected exception Traceback most recent call last File usr local lib python3 8 site packages prefect engine py line 951 in lt module gt enter flow run engine from subprocess flow run id File usr local lib python3 8 site packages prefect engine py line 134 in enter flow run engine from subprocess return anyio run retrieve flow then begin flow run flow run id File usr local lib python3 8 site packages anyio core eventloop py line 56 in run return asynclib run func args backend options File usr local lib python3 8 site packages anyio backends asyncio py line 233 in run return native run wrapper debug=debug File usr local lib python3 8 asyncio runners py line 44 in run return loop run until complete main File usr local lib python3 8 asyncio base events py line 616 in run until complete return future result File usr local lib python3 8 site packages anyio backends asyncio py line 228 in wrapper return await func args File usr local lib python3 8 site packages prefect client py line 81 in with injected client return await fn args kwargs File usr local lib python3 8 site packages prefect engine py line 199 in retrieve flow then begin flow run flow = await load flow from deployment deployment client=client File usr local lib python3 8 site packages prefect client py line 81 in with injected client return await fn args kwargs File usr local lib python3 8 site packages prefect deployments py line 329 in load flow from deployment maybe flow = await client resolve datadoc deployment flow data File usr local lib python3 8 site packages prefect client py line 1695 in resolve datadoc return await resolve inner datadoc File usr local lib python3 8 site packages prefect client py line 1688 in resolve inner data = await self retrieve data data File usr local lib python3 8 site packages prefect client py line 1259 in retrieve data return await storage block read embedded datadoc File usr local lib python3 8 site packages prefect blocks storage py line 79 in read return await run sync in worker thread self read sync key File usr local lib python3 8 site packages prefect utilities asyncio py line 51 in run sync in worker thread return await anyio to thread run sync context run call cancellable=True File usr local lib python3 8 site packages anyio to thread py line 28 in run sync return await get asynclib run sync in worker thread func args cancellable=cancellable File usr local lib python3 8 site packages anyio backends asyncio py line 818 in run sync in worker thread return await future File usr local lib python3 8 site packages anyio backends asyncio py line 754 in run result = context run func args File usr local lib python3 8 site packages prefect blocks storage py line 89 in read sync s3 client download fileobj Bucket=self bucket Key=key Fileobj=stream File usr local lib python3 8 site packages boto3 s3 inject py line 795 in download fileobj return future result File usr local lib python3 8 site packages s3transfer futures py line 103 in result return self coordinator result File usr local lib python3 8 site packages s3transfer futures py line 266 in result raise self exception File usr local lib python3 8 site packages s3transfer tasks py line 269 in main self submit transfer future=transfer future kwargs File usr local lib python3 8 site packages s3transfer download py line 354 in submit response = client head object File usr local lib python3 8 site packages botocore client py line 395 in api call return self make api call operation name kwargs File usr local lib python3 8 site packages botocore client py line 725 in make api call raise error class parsed response operation name botocore exceptions ClientError An error occurred 403 when calling the HeadObject operation Forbidden 11 44 50 180 | INFO | prefect flow runner docker Flow run container celadon cobra has status exited ```

Anna Geller · Answer

Thanks What I meant by moving the traceback was to copy it into the thread while simultaneously removing it from the main message smile if you can do that that would be great Also LMK if the issue persist for some reason after mounting the volume it shouldn t

Azer Rustamov · Answer

Yep I got that just forgot to save the edited message Thank you everything works now