James Wade wrote:
I did a quick look at cpu, the load average is low, but the high cpu utilization is coming from to much iowait time because they are writing to NFS shares.
Not the problem at hand, but you shouldn't suffer that kind of CPU wait unless you are exceeding the capacity of the disk IO on your NFS server, have network issues, or something else. Worthy of more attention if you have the time.
How does Hobbit get CPU utilization? Does it compare idle, user, kernel, iowait, swap? (Say from Top)
So for example, utilization might be 100 - idle time. and take samplings from idle time?
CPU utilization is that: 100 - cpu idle time, as measured by the output of vmstat. Some versions of vmstat report CPU utilization through 4 parameters: user, system, idle, CPU IO wait, others lump iowait and system into a single parameter.
Are the other tools getting their data from vmstat (or sar, etc.)? Personally, I would try watching the server during those timeframes with something like "vmstat 2" and see/understand for myself what is happening. If it happens at odd hours, just cron a little script and analyze the data in the morning. Once you have an idea of what you saw, you can figure out which reporting tool is giving valid results.
Tom
Tom Georgoulias Systems Engineer McClatchy Interactive