If you are using Prefect cloud you could do it either way. You could set them in the Prefect Cloud UI and they would be stored on Prefect’s end in their cloud. Or you could set them locally via a config file or environment variables.
If you are setting them locally. They need to be available during flow execution. This would vary based on the run config. If you are using a LocalRun, it would need to be on the machine with the agent installed. If you are using DockerRun, it would need to be available in the docker container that runs, etc.
https://docs.prefect.io/orchestration/concepts/secrets.html