https://prefect.io logo
Title
m

Miguel Moncada

03/31/2023, 1:29 PM
Hi! is there any built-in operation for
GcsBucket
to delete a blob, or do I need to directly make use of Google's client? 🤔 I couldn't find any after checking here. Thanks a lot!
1
I need to "move" (rename) a blob object, which according to GCP docs it's needed to actually copy it to the destination, then delete the previous blob. (ref). I was planning to directly leverage the
GcsBucket
class if possible for this task, I can see
cloud_storage_copy_blob
but cannot find a way to then remove the "old" blob
k

Kevin Grismore

03/31/2023, 2:17 PM
I actually just did blob renaming via the GcsBucket a few days ago. You can access `google.cloud.storage`'s
Bucket
class from the
GcsBucket
block, then call its
rename_blob
method. Let me grab you some sample code.
👍 2
👀 1
We're looking to rename blobs exported from BigQuery so we're listing blobs that match a certain pattern and doing some silly string manipulations, but something like this should do the job.
@task
def rename_blob(path: str):
    gcs_bucket = GcsBucket.load('your-bucket-block')
    blob_list = gcs_bucket.list_blobs(path)

    blobs_to_rename = list(
        filter(lambda blob: blob.name.split('/')[-1].startswith('0'), blob_list)
    )

    for blob in blobs_to_rename:
        file_name = '_'.join(blob.name.split('/')[-1].split('_')[1:])
        file_path = '/'.join(blob.name.split('/')[:-1])
        gcs_bucket.get_bucket().rename_blob(blob, '/'.join([file_path, file_name]))
m

Miguel Moncada

03/31/2023, 2:21 PM
I wonder why Google's documentation points to copy + delete method if there's a
rename_blob
at all 🤷
k

Kevin Grismore

03/31/2023, 2:21 PM
that might be what
rename_blob
does underneath, I haven't actually looked.
m

Miguel Moncada

03/31/2023, 2:21 PM
thanks a lot @Kevin Grismore :blob-attention-gif:
k

Kevin Grismore

03/31/2023, 2:21 PM
np!