Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

• <@ULVA73B9P> Are there any discounts available on the Team plan if we opt for annual billing instead of monthly?
• We're planning to run the worker via a batch file using NSSM as a Windows service. Would you consider this a reliable, production-ready approach?
From an architectural perspective, would you recommend:
• One VM with one worker pool and two queues (Dev and Prod), *or* Two separate VMs, each with its own pool (Dev and Prod)?
• Are there any best practices you’d recommend specifically for a Windows VM worker setup?
• Our use case includes 24/7 ELT workloads, so we cannot rely solely on serverless execution. Windows VMs are our preferred choice, as our infrastructure team manages patching, monitoring, and resource alerts.


:white_check_mark: thought for 35.9 seconds

Regarding your specific questions:

*Using NSSM as a Windows Service:*
Yes, using NSSM to run the Prefect worker as a Windows service is a production-ready approach. NSSM is widely used and provides several advantages:
- Automatic restart on failure
- Proper service management
- Integration with Windows event logging
- Clean process management

*Architecture Recommendation:*
For your use case, I strongly recommend *two separate VMs with their own pools* (Dev and Prod) rather than one VM with two queues. Here's why:

1. *Environment Isolation:*
   - Prevents resource contention between Dev and Prod workloads
   - Allows independent scaling of resources
   - Reduces risk of Dev activities impacting Prod performance

2. *Maintenance Benefits:*
   - Can perform maintenance on Dev without affecting Prod
   - Easier to implement different backup/restore policies
   - Separate security contexts and access controls

*Windows VM Worker Best Practices:*

1. *Service Configuration:*
   - Use NSSM's failure actions to automatically restart the service
   - Configure appropriate startup delays
   - Set proper service dependencies (if any)

2. *Resource Management:*
   - Configure appropriate process priority via NSSM
   - Set worker concurrency based on VM resources
   - Monitor memory usage and adjust as needed

3. *Operational Considerations:*
   - Implement proper logging paths
   - Use Windows Event Log integration
   - Configure appropriate service account permissions

4. *Worker Configuration:*
```bash
prefect worker start -p "your-pool-name" -t process --limit 100
```
- Use process-based workers for Windows
- Set appropriate concurrency limits based on VM resources
- Consider using tags for workload routing

5. *Monitoring:*
   - Leverage your infrastructure team's existing monitoring
   - Set up Prefect Cloud notifications for worker health
   - Monitor worker queue depths

This setup aligns well with your requirements for 24/7 ELT workloads and leverages your existing infrastructure team's capabilities for VM management. The separation of Dev and Prod environments provides the reliability and isolation needed for production workloads while maintaining operational efficiency.

Would you like more specific details about any of these aspects, particularly around the NSSM service configuration or worker setup?