x

    xyzz

    2 months ago
    Hello, regarding Prefect 2.0: just wondering... are blocks supposed to be the final answer for secrets or will there be further abstractions for supporting e.g. Hashicorp Vault?
    Anna Geller

    Anna Geller

    2 months ago
    Nothing is ever the final answer (other than perhaps :42: ) There are no plans to add any extra Secrets backend because Blocks are so cool, flexible and extensible. But if you already have some Secrets stored in your own Vault, we may add a Block for Hashicorp Vault to make it easy for you to retrieve your existing credentials
    LMK if Hashicorp Vault block is what you're looking for, if so I can open a ticket for backlog
    moving your messages to the thread - next time please keep everything in 1 message or continue adding new messages in a thread: • I'm a bit worried because without Vault support it will be hard getting a Prefect setup in my company • though I guess it might be acceptable to just kind of sync Vault secrets to secrets stored in blocks, as long as user can't look up the value
    Andreas

    Andreas

    2 months ago
    +1 for adding a Block for Hashicorp Vault. This could be usefull in the future for some of the projects I am working on too
    Anna Geller

    Anna Geller

    2 months ago
    @Marvin open "Add a Block for retrieving Secrets from Hashicorp Vault"
    Marvin

    Marvin

    2 months ago
    x

    xyzz

    2 months ago
    yes I was talking about Hashicorp Vault, thanks for opening the ticket!
    davzucky

    davzucky

    2 months ago
    @Anna Geller in the future would block be part of the collection pattern idea or splitting dependency?
    Anna Geller

    Anna Geller

    2 months ago
    the Block itself configuring credentials would likely be in the core package and configurable through UI, but to perform some business logic/action with it, you'll likely have to install a Collection cc @alex
    davzucky

    davzucky

    2 months ago
    Interested to understand how would that work @alex
    alex

    alex

    2 months ago
    @davzucky Any set of blocks created to interact with Hashicorp vault would live in a Prefect Collection. We have mechanisms in place that make any blocks in installed Prefect Collections available for configuration in the UI. By default, all integrations should go in a collection so that we can minimize the number of dependencies that the core library has.
    davzucky

    davzucky

    2 months ago
    Great thank you for the reply. Looking forward to see a sample collection fully implemented with this feature
    alex

    alex

    2 months ago
    Here’s a PR where I’m converting an existing dataclass to a block in a Collection as a reference: https://github.com/PrefectHQ/prefect-dbt/pull/26
    x

    xyzz

    2 months ago
    Just wondering... is there anything I can do to improve the chances of this being implemented?
    I'm still a prefect newbie, so I'd be a bit worried about implementing it myself
    and didn't work directly with Vault yet either
    Anna Geller

    Anna Geller

    2 months ago
    we added it to the roadmap and you can track via GitHub issue - if this is critical for you, LMK - I can ask the product team if we can prioritize it
    x

    xyzz

    2 months ago
    Thanks, Anna!
    At the moment it isn't critical since we are still at the evaluation phase, but if we go for prefect this is sometehing we'll definitely need.