If system is busy, deny request at the earliest entry point (e.g. if database is busy then stop accepting requests at the web server)
If under load stop accepting requests and
- process pending requests or
- drop them all or
- restart application
If adjusting system make sure you're not just moving the load to some other component (e.g from CPU->HDD)
Things to check for
Java:
- Pools size
- Heap utilization
- Garbe collection
- Cache hit ratio
- Messaging systems
Communication:
- Network I/O (bytes r/w, throughput)
- Disk I/O
- CPU usage
- Memory usage
- Threads usage