Hi all! I see some inconsistencies in terms of task results when using S3 storage (script based) vs Docker storage and wanted to double check if I’m missing something here. My flow looks roughly like this:

def get_flow(...):
  with Flow(...) as f:
    tasks
  return f

then I use it in another file for deployment and set some env-specific configuration:

from module_x import get_flow

f = get_flow()
f.result = S3Result(bucket=dev or prod bucket here)

# no results with this storage configuration
f.storage = S3(bucket & key, stored_as_script=True, local_script_path=<this file>)

# results work with this storage
f.storage = Docker(...)

f.register()

With S3 storage I don’t get any results saved in S3, everything works fine when using Docker storage. I wonder if this has anything to do with Docker using serialization and S3 using the “as script” approach

can you share the storage configuration? 🙂

sorry I messed up with new lines and my message was posted too early 😅 now the example is complete

Yes that is related. This happens because

flow.register()

gets metadata and pushes it to Prefect cloud. The metadata can be found here. Notice executor and results are not part of it so we don’t store that. I think because result can contain sensitive info like the bucket. So it gets loaded in from the Flow file so you need to define it there

ah that makes more sense now! so I assume when I use Docker storage, it works because the flow gets serialized into the storage including the result (because the entire flow Python object is serialized), but when I use script-based S3 storage, all the dynamic settings not directly defined in

get_flow

are not picked up. Would this reasoning be correct?

Yes exactly

👍 thanks for clarifying, that was a bit confusing, I tested S3Storage with

stored_as_script=False

and the results work now