@Marvin Is it safe to have a single bucket for all of my flows result_storage or should there be separate buckets and/or folders ?

Here's what I found from the Prefect documentation regarding using a single bucket for all flow result storage: 1. Result Storage Configuration: Generally, Prefect uses the result storage configured on the flow unless otherwise specified. If no storage is explicitly configured, results are stored locally by default. Source 2. AWS S3 Storage: You can configure a storage block from a cloud integration library like

prefect-aws

. This allows result storage specific to your configuration. Source While Prefect does not explicitly state if a single bucket is inherently unsafe, organizing separate buckets or folders might be a good practice for clarity, access control, and to avoid data clashing. For personalized advice based on your specific use case, you can explore the Prefect forums or repositories. Refer to the official docs for more detailed guidance: - Results Concept: https://docs.prefect.io/latest/concepts/results/ - Moving Data: https://docs.prefect.io/latest/guides/moving-data/

i would say its safe as you long as you make use of

result_storage_key

to namespace the bucket with some paths so that different types of writers dont step on each others toes like this example from the docs

from prefect import flow, task

@flow()
def my_flow():
    hello_world()
    hello_world(name="foo")
    hello_world(name="bar")

@task(persist_result=True, result_storage_key="hello-{parameters[name]}.json")
def hello_world(name: str = "world"):
    return f"hello {name}"

my_flow()

except you could add a prefix to help with namespacing

@task(persist_result=True, result_storage_key="{PREFIX}/hello-{parameters[name]}.json"