https://prefect.io logo
s

Sean Turner

09/26/2022, 7:42 PM
Hey all, trying to read a file with the
RemoteFileSystem
.
Copy code
rfs = RemoteFileSystem(basepath="<s3://bucket-name/some/path/on/bucket>")
cfg = rfs.read_path("base_config.yaml")
I end up with
cfg
being a
coroutine
. I imagine this returns bytes but there aren't any helper methods and it's not clear how I'm supposed to proceed? I am interested in reading this file into memory.
1
n

Nathaniel Russell

09/26/2022, 7:45 PM
That function is asyncronous so something like this is required
Copy code
cfg = asyncio.run(rfs.read_path("base_config.yaml"))
🙌 1
z

Zanie

09/26/2022, 7:46 PM
I think these should have a
sync_compatible
annotation since 2.4.2
(which will automatically run it in an event loop for you)
🙌 1
s

Sean Turner

09/26/2022, 7:52 PM
Thank you Nathaniel This ends up working best I think, your suggestion throws a warning about how
RemoteFileSystem.read_path
was never awaited. Never really done async await especially in python though. I know go concurrency
Copy code
cfg = rfs.read_path("base_config.yaml")
my_yaml = await(cfg)