@Marvin If I try to deploy this code:

from prefect import flow
from prefect.logging import get_run_logger
from prefect.filesystems import RemoteFileSystem

import pystac_client

from utils_2.utils_2_fun import hello_utils_2

a = RemoteFileSystem.load("test-remote-fs")

@flow(flow_run_name="flow2_main", result_storage=a)
def flow2_main() -> str:
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"{hello_utils_2()} String from Flow 2: pystac_client {pystac_client.__version__}")
    return f"{hello_utils_2()} String from Flow 2: pystac_client {pystac_client.__version__}"

I get the following error:

prefect.exceptions.ScriptError: Script at 'src/flow2/flow2_main.py' encountered an exception: TypeError('Result storage configuration must be persisted server-side. Please call `.save()` on your block before passing it in.')

Note that the RemoteFileSystem "test-remote-fs" has already been saved, and in fact it can be seen in the Prefect UI. I've already tried calling the

save

method on

a

(the RemoteFileSystem) but it does not solve the problem.

It looks like you're encountering an issue with setting up result storage for your flow using

RemoteFileSystem

. The error suggests that the result storage configuration isn't properly persisted server-side, even though you saved it as a block. Here are a few key points and steps to consider, based on Prefect's documentation: 1. Ensure Correct Saving of Blocks: When using the

RemoteFileSystem

, make sure you actually saved the block with a specific name before using it in your flow. According to Prefect's 2.x and 3.x documentation, the pattern for storing this would typically look like:

python
   from prefect.filesystems import RemoteFileSystem

   block = RemoteFileSystem(basepath="<s3://my-bucket/my-folder/>")
   block.save("test-remote-fs")  # Ensure this is executed

2. Verify Block Existence: Double-check in the Prefect UI if the block named

"test-remote-fs"

is correctly listed under the storage blocks section. 3. Version-Specific Details: - For Prefect 2.x and 3.x, ensure you are using the block's name correctly and that the block is saved and retrieved properly. Here are the links to the relevant documentation for deeper insights: - Prefect 2.x RemoteFileSystem - Prefect 3.x Result Storage If everything seems in order and you're still facing issues, ensure all systems are properly authenticated and there are no network restrictions affecting access to your remote file system. Don't worry, it's not that Prefect has a grudge against you; sometimes it just wants a bit of extra reassurance with its storage settings!

I'm using Prefect 3. The code you provide did not work, I get the same issue.

Ah, I see why that might be frustrating. Let's dive a bit deeper into the nuances of Prefect 3 when it comes to result storage. 1. Persistent Storage Default: Ensure you have configured the default result storage in your Prefect settings. If using the

RemoteFileSystem

, ensure it's properly set up with not just

.load()

, but that it's the explicit block you want to use: - You can specify a default by using the

PREFECT_DEFAULT_RESULT_STORAGE_BLOCK

setting. This ensures Prefect knows where to persist the results if you haven't explicitly specified a block in the flow or tasks. 2. Result Storage Key: Consider specifying a

result_storage_key

to ensure Prefect knows exactly where and how to store results. This can be done by using a templated string to specify unique filenames or paths for your results. 3. Ensure Persistence is Enabled: Make sure that persistence is enabled for your tasks or flows. This involves using the

persist_result

parameter in the task or flow decorators. Given these considerations, you might need to review your configuration against the Prefect results documentation for version 3.x: Prefect 3.x Result Storage. By making sure that default settings and result storage configurations are correctly set up, you should be able to resolve the

TypeError

you're encountering. Remember, even Prefect sometimes just needs a gentle nudge in the right direction!

@Marvin prefect_aws doesn't have an S3 class

It seems that the

prefect_aws

module doesn't include a distinct

S3

class, but rather provides functionality through the

S3Bucket

class. This class is designed to work with both AWS and MinIO credentials. The

S3Bucket

class uses

AwsCredentials

and

MinIOCredentials

for authentication. Here are some relevant points: - The

S3Bucket

class can handle both AWS and MinIO credentials using Pydantic models. - If you're facing issues with MinIO credentials, ensure that you use the correct credentials field (

MinIOCredentials

) when initializing the

S3Bucket

. For more details, you might want to look at the GitHub repository for prefect-aws or check out specific issues like this one if you encounter credential-related errors. If you need any specific help with using the

S3Bucket

class, feel free to ask!