Not sure if this is related to my issue about Whoops errors or not, but it seems that the history does not always update correctly when a service goes down.
Seems to be the same problem as here...
http://www.hswn.dk/hobbiton/2008/03/msg00064.html
My history.log shows no issues... Bunch of "Will not update <server.test> - color unchanged" messages.
hobbitlaunch.log shows this.
2008-03-26 23:21:05 Whoops ! bb failed to send message - timeout 2008-03-26 23:56:17 Whoops ! bb failed to send message - timeout
-- Stewart
The revolution will not be televised. The revolution will be no re-run brothers; The revolution will be live.
On Thu, Mar 27, 2008 at 04:48:00PM -0400, Stewart L wrote:
Not sure if this is related to my issue about Whoops errors or not, but it seems that the history does not always update correctly when a service goes down.
Seems to be the same problem as here...
http://www.hswn.dk/hobbiton/2008/03/msg00064.html
My history.log shows no issues... Bunch of "Will not update <server.test> - color unchanged" messages.
This is not a problem if the errors appear after restarting Hobbit.
History updates are normally only logged when the color of a status changes. However, during a restart color changes might happen while the history module is not up and running, so the first status that Hobbit receives after a restart is always sent to the history module. This message is then logged when the history module detects that the color hasn't changed, and it drops the update.
Since updates will usually trickle in from clients during the first 5-10 minutes of Hobbit running, you will see this for a few minutes after a restart.
Of course, if this happens without Hobbit having been restarted, it is more interesting.
Regards, Henrik
ok, so I'm not worried about the "Will not Change" errors. How about the Whoops errors? What could cause them?
Like I said before, I applied a patch you gave me about non-blocking sockets to try to get rid of this. Is there a way to increase the timeout period? This box is under a fairly heavy load at times. Load average stays a constant 3.0 and peaks up to 10-12. And if I do a top, I show a lot of CPU time spent in iowait.
$ nice top 06:58:13 up 102 days, 20:31, 1 user, load average: 3.40, 3.58, 3.60 149 processes: 148 sleeping, 1 running, 0 zombie, 0 stopped CPU states: cpu user nice system irq softirq iowait idle total 0.0% 0.0% 0.4% 0.1% 0.0% 42.4% 56.8% cpu00 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% cpu01 0.9% 0.0% 0.9% 0.0% 0.0% 98.0% 0.0% cpu02 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% cpu03 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% cpu04 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% 0.0% cpu05 0.0% 0.0% 0.0% 0.0% 0.0% 44.1% 55.8% cpu06 0.0% 0.0% 0.0% 0.0% 0.0% 36.2% 63.7% cpu07 0.0% 0.0% 0.9% 0.0% 0.0% 62.3% 36.6% cpu08 0.0% 0.0% 0.9% 0.0% 0.0% 0.0% 99.0% cpu09 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% 0.0% cpu10 0.0% 0.0% 0.9% 0.0% 0.0% 0.0% 99.0% cpu11 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% cpu12 0.0% 0.0% 1.9% 1.9% 0.0% 96.0% 0.0% cpu13 0.0% 0.0% 0.0% 0.0% 0.0% 43.5% 56.4% cpu14 0.0% 0.0% 0.9% 0.0% 0.0% 35.6% 63.3% cpu15 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% Mem: 16411504k av, 15428124k used, 983380k free, 0k shrd, 168924k buff 7224252k actv, 5861544k in_d, 981512k in_c Swap: 2096472k av, 0k used, 2096472k free 14649352k cached
Stew
On Fri, Mar 28, 2008 at 5:34 AM, Henrik Stoerner <henrik at hswn.dk> wrote:
On Thu, Mar 27, 2008 at 04:48:00PM -0400, Stewart L wrote:
Not sure if this is related to my issue about Whoops errors or not, but it seems that the history does not always update correctly when a service goes down.
Seems to be the same problem as here...
http://www.hswn.dk/hobbiton/2008/03/msg00064.html
My history.log shows no issues... Bunch of "Will not update < server.test> - color unchanged" messages.
This is not a problem if the errors appear after restarting Hobbit.
History updates are normally only logged when the color of a status changes. However, during a restart color changes might happen while the history module is not up and running, so the first status that Hobbit receives after a restart is always sent to the history module. This message is then logged when the history module detects that the color hasn't changed, and it drops the update.
Since updates will usually trickle in from clients during the first 5-10 minutes of Hobbit running, you will see this for a few minutes after a restart.
Of course, if this happens without Hobbit having been restarted, it is more interesting.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Stewart
The revolution will not be televised. The revolution will be no re-run brothers; The revolution will be live.
participants (2)
-
henrik@hswn.dk
-
stewartl42@gmail.com