@Marvin How do I restart pending jobs automatically if they have been pending for too long

There isn't a direct feature that addresses restarting pending jobs that have been enqueued for too long automatically, according to the open enhancements on the Prefect GitHub repository. However, a related issue has been reported, where a flow remains in a "Running" state when the agent is down (GitHub issue #7239), which shows that there are potential edge cases in flow state management that could relate to your question. For the behavior you desire, you would likely need to implement a custom solution that monitors the state of your jobs and restarts them if they've been in the pending state for too long. This might involve the use of Prefect's state handlers or event hooks to track the state of your flows and runs, and the orchestration API or CLI to restart the jobs as needed. These mechanisms allow you to register functions that get called when the state of a task or flow changes, which you could use to automate the restart of pending jobs. Since this kind of script or extra service would need to be built specifically for your workflows, I would suggest checking the Prefect documentation and community discussions for best practices on implementing such a feature.