https://prefect.io logo
Channels
Powered by
w

Will

09/04/2023, 8:55 AM
Hi, I have this MR failing unit testing with the folowing error message: 
botocore.exceptions.NoCredentialsError: Unable to locate credentials
I think it's because of this line referencing S3, but cannot say for sure. I though I'd reach out the community before requesting another review.
@Nate sorry to drag you in, I met you in last month's prefect course. Any suggestions here?
n

Nate

09/06/2023, 2:55 PM
hi @Will - my feeling is that if this is something we're going to test, we would want to mock out any call to a remote filesystem. there should not be any need to actually talk to s3 from our unit tests
actually this test might be instructive https://github.com/PrefectHQ/prefect/blob/main/tests/test_filesystems.py#L375 seems like we could use a 
RemoteFileSystem
in memory like that instead of worrying about mocking
w

Will

09/06/2023, 3:29 PM
Thanks for your response, I just successfully tested locally and was just about to message you to approve the tests for review: https://github.com/PrefectHQ/prefect/pull/10404
n

Nate

09/06/2023, 3:30 PM
oh nice, and you used it in memory too - jinx! 🙂 👀
😅 1
w

Will

09/06/2023, 3:31 PM
Thank you!
Hi @Nate 👋 I pushed some changes on Thursday, any chance you might be able to kick off a review?
n

Nate

09/08/2023, 3:38 PM
👀
im a bit confused about your recent change
Copy code
-    fs = RemoteFileSystem(basepath="<memory://root>")
+    fs = LocalFileSystem(basepath="memory/root")
memory/root
is not a valid 
block-type/block-document-name
, nor is it a valid 
basepath
that specifies a protocol i would think the eventual implementation would have 
PREFECT_DEFAULT_RESULT_STORAGE_BLOCK
accepting a block type/document slug, as the name suggests, like 
s3/my-result-storage
and then maybe this would be more like
Copy code
return (
        Block.load(PREFECT_DEFAULT_RESULT_STORAGE_BLOCK.value())
        if PREFECT_DEFAULT_RESULT_STORAGE_BLOCK.value() is not None
        else LocalFileSystem(basepath=PREFECT_LOCAL_STORAGE_PATH.value())
    )
Powered by