Well, the server was restarted back on the 4th... Current procs: hobbit 12332 12230 0 Mar 04 ? 5:26 hobbitd_channel --channel=page --log=/var/log/hobbit/page.log hobbitd_alert --c hobbit 12231 12230 2 Mar 04 ? 379:51 hobbitd --pidfile=/var/log/hobbit/hobbitd.pid --restart=/opt/home/hobbit/server hobbit 12326 12319 0 Mar 04 ? 40:04 hobbitd_rrd --rrddir=/opt/home/hobbit/data/rrd hobbit 12328 12321 0 Mar 04 ? 6:02 hobbitd_client hobbit 12333 12330 2 Mar 04 ? 117:51 hobbitd_history hobbit 12321 12230 0 Mar 04 ? 1:44 hobbitd_channel --channel=client --log=/var/log/hobbit/clientdata.log hobbitd_c hobbit 12331 12230 0 Mar 04 ? 0:15 hobbitd_channel --channel=clichg --log=/var/log/hobbit/hostdata.log hobbitd_hos hobbit 12334 12331 0 Mar 04 ? 0:25 hobbitd_hostdata hobbit 12327 12320 0 Mar 04 ? 4:43 hobbitd_rrd --rrddir=/opt/home/hobbit/data/rrd hobbit 12330 12230 0 Mar 04 ? 0:04 hobbitd_channel --channel=stachg --log=/var/log/hobbit/history.log hobbitd_hist hobbit 12320 12230 0 Mar 04 ? 0:12 hobbitd_channel --channel=data --log=/var/log/hobbit/rrd-data.log hobbitd_rrd - hobbit 12319 12230 0 Mar 04 ? 21:28 hobbitd_channel --channel=status --log=/var/log/hobbit/rrd-status.log hobbitd_r hobbit 12335 12332 1 Mar 04 ? 153:22 hobbitd_alert --checkpoint-file=/opt/home/hobbit/server/tmp/alert.chk --checkpo
hobbitd_history seems to be running...
Again, status of other tests are being reflected properly, history recorded properly, both for the same test on other hosts AND for different tests on the same host. It's like random histories are being ignored... x,x
history.log shows some errors... But they are later than confirmed times when some histories weren't being recorded (~2 days ago).
hobbitlaunch.log shows a termination of hobbitd 2 days ago... But this was the time of a restart so perhaps it's expected. Again, it has been writing valid histories for some items since then. I'll try forcibly shutting down the server and brining it back up as well.
stephen
-----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Thursday, March 06, 2008 1:43 PM To: hobbit at hswn.dk Subject: Re: [hobbit] 'red' status not showing up in history
On Thu, Mar 06, 2008 at 10:56:16AM -0800, Menton, Stephen wrote:
I have a test that's gone 'red' multiple times today. I know this because I've seen it in the website and alerts have been sent to me. Yet if I look in ~hobbit/data/hist, ~hobbit/data/histlogs, or use the hsitory CGIs in the web GUI, it shows that it hasn't been 'red' today,
rather that it's been green for >2 days.
The only explanation I can give is that the hobbitd_history module might have been stopped or crashed when this red status happened. Could you check the history.log and hobbitlaunch.log files to see if there's any mention of this ?
Doing a full restart of Hobbit will force a sync of the history logs with the current status recorded in Hobbit, but it obviously cannot record events that are long past.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk