Hi guys can I use S3 block to download files from S3 bucket

Question

Hi guys can I use S3 block to download files from S3 bucket or am I just using the block as secret to the AWS S3Download task any examples

Anna Geller · Answer

Yes totally both works up to you which one you prefer

Anat Tal Gagnon · Answer

Thanks Anna You got a code snip as example

Anna Geller · Answer

```from prefect filesystems import S3 s3 block = S3 load dev s3 block put directory local path s3 path ```

Anna Geller · Answer

just an example

Anna Geller · Answer

you can inspect the block methods in code to see more how this works docstrings are quite good

Anat Tal Gagnon · Answer

< Anna Geller> I m getting an error RuntimeWarning coroutine S3 put directory was never awaited

Anat Tal Gagnon · Answer

Untitled py

Anna Geller · Answer

You d likely need await in front of it

eddy davies · Answer

Is generating a local file and then pushing it to s3 as shown here the best practice Rather than saving a file straight to s3

eddy davies · Answer

I tried to create a super basic attempt at using await but it gives a massive <https pastebin com 4YhRpbqd|error>

Anna Geller · Answer

What are you trying to do Eddy

eddy davies · Answer

Save data to S3 though I do notice now that the start of this thread was downloading a file instead

Anna Geller · Answer

this works for me need to check how to use it with Blocks ```import fsspec fs = fsspec filesystem s3 fs put Users anna repos dataflow ops docs <s3 prefect orion docs> recursive=True ```

eddy davies · Answer

Great that works I am not providing any AWS Access Key is it using that from my block somehow then

Anna Geller · Answer

from your ~ aws credentials

eddy davies · Answer

how can I guarantee this will run when deployed too production say using dask docker and or kubernetes