I'm working on something similar where we don't want our secrets storage duplicated for different use cases. We're using Google Secret Manager so I think my approach is to use
GcpSecret blocks in Prefect that essentially just point to the Secret in Secret Manager.
I'm not sure if it can be referenced within the yaml but maybe this is helpful.