o

    Omar Sultan

    1 year ago
    Hi Guys, We're looking to use Webhook Storage to store the flows in our local environment. The Prefect server will be hosted in a K8 Cluster, any recommendations for a storage service that would fit that requirement?
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Omar Sultan, I haven’t really seen people use Webhook storage that much unless they already have their own file storage system. The example in the docs shows Dropbox but let’s see what the community has to say
    o

    Omar Sultan

    1 year ago
    HI Kevin, the challenge I am facing is that I want to implement this in a closed environment that is not connected to the internet. We did some experiments between Script and Pickle storage and decided that Pickle storage is better. So I am now exploring options that would work for us with this setup
    Hopefully someone from the community will have some ideas 😄
    Kevin Kho

    Kevin Kho

    1 year ago
    But why would you use the webhook then? Doesn’t Local storage make the most sense?
    o

    Omar Sultan

    1 year ago
    If I understand correctly local storage won't work if I am registering the Flow from a remote client. Currently the setup is as follows: • Prefect Running on Kubernetes using Helm • Flows run using Kubernetes Agent • Data Engineers can build their flows and register them remotely from their laptops to be scheduled and run on the server. So local storage would end up storing it on the engineer's laptop •
    Kevin Kho

    Kevin Kho

    1 year ago
    You might be able to use local storage, which would make the path relative to the Docker container
    o

    Omar Sultan

    1 year ago
    I'll check that option out
    Thanks Kevin