Thread
#prefect-community
    Vamsi Reddy

    Vamsi Reddy

    10 months ago
    Hi all, we are using the s3 storage to store our flows. currently they are all under one s3 bucket. we would like to store then under the same bucket but under seperate sub directory like
    bucketname/dev
    and
    bucketname/prod
    and have our dev and prod flows stored accordingly. i am using
    storage=S3(bucket="bucketname")
    but would like to have something like
    storage=S3(bucket="bucketname/dev")
    . Any idea how to do this?
    @Kevin Kho any thoughts on how to do this ?
    Kevin Kho

    Kevin Kho

    10 months ago
    Wouldn’t it be in the file path instead of the bucket name?
    Vamsi Reddy

    Vamsi Reddy

    10 months ago
    where do we specify the file path
    ?
    *_class_ prefect.storage.s3.S3*(bucket, key=None, stored_as_script=False, local_script_path=None, client_options=None, upload_options=None, **kwargs)
    Derek Heyman

    Derek Heyman

    10 months ago
    @Kevin Kho When you specify the flow name it get's sluggified so it doesn't maintain any slashes that would result in sub-buckets like we're looking for
    Kevin Kho

    Kevin Kho

    10 months ago
    Wouldn’t you supply it with the
    key
    ?
    Or does the key slugify things Derek?
    Vamsi Reddy

    Vamsi Reddy

    10 months ago
    so the key uploads individual flows. what we really want is all our dev flows to be stored under a folder bucket_name/dev/ all dev flows. however with the key it uploads the flow as bucket_name/dev_flow dev_flow is the key here
    Kevin Kho

    Kevin Kho

    10 months ago
    Will try this in a bit
    Vamsi Reddy

    Vamsi Reddy

    10 months ago
    ok i tried something which did the trick but not sure if its the right approach: storage=S3(bucket=“bucket_name”, key=“dev/xyz_flow_1.0.1")
    this creates the dev directory and adds the flow with the exact same name not sluggified to it
    Kevin Kho

    Kevin Kho

    10 months ago
    yeah I just finished trying, and that’s what I meant. I think this is the only way to achieve it because it’s really not in the bucket name
    Vamsi Reddy

    Vamsi Reddy

    10 months ago
    thank you 😎