https://prefect.io logo
Title
l

Lucien Fregosi

09/07/2022, 10:31 AM
Hi 👋 I’m using GCS Bucket as a block filesystem for Prefect 2.0 and I wonder if it’s possible to clean this bucket ? Indeed when I delete a file from my python code, the file stays on the bucket and it is still taken/executed at the next prefect run. Any inputs/idea on that ?
1
j

Jeff Hale

09/07/2022, 1:32 PM
Hi Lucien. Prefect doesn’t have a command to delete a file in AWS. However, Prefect is using s3fs under the hood, so you could delete file in your Python code. Here’s an example: https://stackoverflow.com/a/66347767/4590385
l

Lucien Fregosi

09/07/2022, 1:35 PM
Ok thanks for your reply @Jeff Hale You mean I can execute this python code in the same file as the block creation/registration and force the block to be redeployed each time the applicative code change ?
j

Jeff Hale

09/07/2022, 2:54 PM
Or you could make another flow that uses this code to clean up the bucket.