Hi all, I might be missing it but, is there a way to add overwrite=True to the Prefect Azure BlobStorageUpload method?

@Jelle Vegter I’m no Azure expert, but object storage (S3, GCS) normally works this way: if you upload a specific file for the first time, it will create an object storage object in a specified bucket (or for Azure: container) and location. If you redo the operation, the existing object (file) will be replaced by the new one. This means that there is no need to pass something like overwrite=True. If an object already exists, it will be automatically overridden.

In Azure this is not the case. This is the error: azure.core.exceptions.ResourceExistsError: The specified blob already exists. RequestId:5bef0cf1-b01e-002e-6

I see. Azure’s blob storage seems to be really “special” 🙂 Feel free to submit a PR. Alternatively, you can use your own custom task, or leave blob_name empty - then the file will be stored with a random uuid as name (this wouldn’t overwrite, but create a new file each time).

You are right. You’d need to change this like to take in

overwrite=True

for now. We can open an issue though

Thanks! Untill then I'll continue to use my own functions