I’m using GCS Bucket as a block filesystem for Prefect 2.0 and I wonder if it’s possible to clean this bucket ?
Indeed when I delete a file from my python code, the file stays on the bucket and it is still taken/executed at the next prefect run.
Any inputs/idea on that ?
09/07/2022, 1:32 PM
Hi Lucien. Prefect doesn’t have a command to delete a file in AWS. However, Prefect is using s3fs under the hood, so you could delete file in your Python code.
Here’s an example: https://stackoverflow.com/a/66347767/4590385
09/07/2022, 1:35 PM
Ok thanks for your reply @Jeff Hale
You mean I can execute this python code in the same file as the block creation/registration and force the block to be redeployed each time the applicative code change ?
09/07/2022, 2:54 PM
Or you could make another flow that uses this code to clean up the bucket.