Hello, I am struggling to get the s3 bucket filesy...
# ask-community
k
Hello, I am struggling to get the s3 bucket filesystem to work. I am trying to convert from remote-file-system storage to s3 bucket storage. I would like to have one bucket with all my flows. Since the bucket contains other items, all the flows should be stored in the "storage" directory, whose path looks like this:
Copy code
<s3://tps-prefect/storage/>
My block definition for my flow reflects this in the first image I am attaching. I then execute the following deployment command
Copy code
prefect deployment build -sb "s3-bucket/ice-harvester-block" -n "ice-harvester-flow-prod" -q "default" "flow.py:ice_harvester_flow"
This works as I would expect, and all my flow code gets uploaded to the path
Copy code
<s3://tps-prefect/storage/ice-harvester-flow/>
however, when I run the deployment, I am getting the following error:
Copy code
FileNotFoundError: [Errno 2] No such file or directory: '/app/flow.py'
Where is the "app" coming from? I am assuming it is the name of the folder prefect creates in my docker to store my code, which is fine. But if this is the case then if the deployment is pulling code based on my block path, then flow.py should be in app, since it is in the blocks defined path mentioned above. I am confused of how this works. If the deployment uses the block info to write correctly, how is it using the same block and reading the info incorrectly? Does anyone know why this behavior is occurring?
Okay, so i figured out a work around. If you remove the bucket folder path from the block and instead define it in the deployment command with the --path param, it fixes this. Not sure why path isn't defaulting to the bucket folder since it is defined in the block, maybe this is intentional for some reason