We do a lot of downloading and un-zipping of zip files. Does anyone have any tricks or methods for doing this in Prefect server? I think the challenge is the unzip process.
01/11/2022, 3:05 PM
Do you mean the challenge is executing it? Or is it like slow? For execution, it’s just
with zipfile.ZipFile(path_to_zip_file, 'r') as zip_ref:
01/11/2022, 4:04 PM
Thanks, Kevin. More of an implementation challenge: Where would wget or the request object save the file and where would zip_ref extract all the zipped files. I know a tiny bit about Storage objects, but that seems like overkill for temporary storage space in this example...
01/11/2022, 4:06 PM
I think just in the working dir of the Flow? Yep you shouldnt need storage