hi - in my test environment I need to access my S3 Storage (mocked out with Localstack) via a different base URL depending on if I’m registering the container (via my local command line) or actually running it (in local Kubernetes). I’ve tried to configure things like this:

home = os.environ.get("HOME")
if home != "/Users/ryan.sattler":
    s3_url = "<http://localstack:4566>"
else:
    s3_url = "<http://localhost:31566>"

flow.storage = S3(bucket="prefect-flows", key="hello-task.py", client_options={
    "endpoint_url": s3_url,
    "aws_access_key_id": "",
    "aws_secret_access_key": ""
})

However this doesn’t seem to work (the localhost url is always being used, so it works for registration but not at runtime), possibly because the value of flow.storage is getting baked-in at registration time? How can I make this dynamic? Unfortunately given my company’s security setup there is no convenient way to use a real S3 bucket (or real container registry or etc) when testing locally.

Hi Ryan - by default the Flow object is converted to static bytes and deserialized when run, which is why you're seeing a static `s3_url`; to resolve this, you'll want to provide two kwargs to your S3 storage class: •

stored_as_script=True

to specify that you'd like to store your flow's definition script •

local_script_path

- the full path to your Flow file that defines your flow; this file will be uploaded to your "S3" bucket With this configuration, the file will be re-parsed at runtime which should pick up your dynamic logic

have made those changes but still getting the same error:

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "<http://localhost:31566/prefect-flows/hello-task.py>"

Oh I see, my apologies; yea the s3 URL is stored in the backend and is referenced by your agent to download the flow so this is trickier than I suggested

No worries. I got it working by hackily registering the flow from another k8s pod I had lying around. That’ll be good enough for now (just testing things out) - thanks

For sure -- this is definitely an interesting new edge case for me; we have some large changes in the works that I believe will alleviate this sort of chicken / egg problem quite nicely, but it's going to be a while before it's all rolled out so stay tuned 😄