Thread
#prefect-community
    r

    rectalogic

    2 months ago
    what's best practice for dealing with Storage? We have our flows in github and for testing we want them to run locally in checked out repo using
    Local
    Storage, but then for production we want the same flow to run on our ECS agent using a
    GitHub
    storage pointing to the same repo. With the
    UniversalRun
    config it can run on either agent, but how do we make the storage conditional so it uses the correct storage depending on which agent it's targetted to?
    Kevin Kho

    Kevin Kho

    2 months ago
    This can’t be done because you need storage to be defined at registration time so this will take two separate registrations. This is part of the reason storage was decoupled from deployment
    r

    rectalogic

    2 months ago
    so how is this typically handled? Define two Flows for each set of tasks, one with Local and one with GitHub? Or edit the source code to test with a local agent vs ECS agent?
    Kevin Kho

    Kevin Kho

    2 months ago
    I think something like that. You can define a flow in one script with no storage. And then import it from a second script and attach the storage and run config and then call register. So you can decouple the reregistration script and that makes things a bit better
    So your registration script can take parameters through CLI that then push it to dev or prod.
    r

    rectalogic

    2 months ago
    to attach a storage to a flow later, should I directly access the attribute
    myflow.storage = GitHub()
    or do I add the flow to the storage
    GitHub().add_flow(myflow)
    ?
    Kevin Kho

    Kevin Kho

    2 months ago
    Exactly, just the first one
    r

    rectalogic

    2 months ago
    thanks
    Kevin Kho

    Kevin Kho

    2 months ago
    Just note executor specifically needs to be defined in the flow itself because it’s not serialized along with the Flow. RunConfig, Storage are fine to attach this way