I have setup docker and Prefect on remote on-premise server. How should I keep the Prefect server with docker agent always running?

Yes it needs to be on to orchestrate things

Can this help keep the server on local machine running https://docs.prefect.io/orchestration/server/deploy-local.html#ui-configuration

You can run the agent in detached mode with

-d

. Yes supervisor for the agent