If I’m understanding you correctly, then yes, that’s possible. We have a different, vault specific, block that you can use:
https://github.com/pbchekin/prefect-vault
Essentially, instead of telling your secret to Prefect, you tell Prefect where to find the secret you want in Vault.