Hi,
We are running a new installation of Hobbit 4.2 on Solaris 10 running in a non-global zone. Server is a v240 but I don't think that matters here.
The problem here is that our bbgen status turns yellow with fairly high frequency, sometimes multiple times an hour, at (what seem like) random intervals. In the yellow alert bbgen reports: "hobbitd status-board not available"
During this time the hobbitd daemon is still running and the next time that bbgen runs the alert (usually) turns green. I've tested this by running bbgen every second, every 15 seconds, and every minute. The same is also true if I run bbgen by hand.
During the 'yellow alert' time window the bb2.html gets updated with "All Monitored Systems OK" When all monitored systems are NOT OK. When the status turns green again this page reflects the correct status for the non-green systems.
Below are the output from some commands/logs. These logs don't really seem to help, so let me know if there is anything else that I can send along to debug this issue.
Any help is appreciated - we're near the point of frustration to where we may have to pull the plug on Hobbit and go back to our old BB installation.
Thanks in advance. -Jon
(logs below)
hobbitd log from --debug. Way less entries here than normal. 2006-11-03 10:54:00 -> do_message/1 (12 bytes): hobbitdboard 2006-11-03 10:54:00 -> update_statistics 2006-11-03 10:54:00 <- update_statistics 2006-11-03 10:54:00 -> oksender 2006-11-03 10:54:00 <- oksender(1-a) 2006-11-03 10:54:00 -> setup_filter: hobbitdboard 2006-11-03 10:54:00 <- setup_filter: hobbitdboard 2006-11-03 10:54:00 <- do_message/1 2006-11-03 10:54:01 -> do_message/1 (0 bytes): 2006-11-03 10:54:01 -> update_statistics 2006-11-03 10:54:01 <- update_statistics 2006-11-03 10:54:01 <- do_message/1
$BB --debug $BBDISP "hobbitdboard" (with no --debug on a 'failure' I get no output. I'm assuming this is the same cause of the bbgen yellow alert)
2006-11-03 10:54:01 Transport setup is: 2006-11-03 10:54:01 bbdportnumber = 1984 2006-11-03 10:54:01 bbdispproxyhost = NONE 2006-11-03 10:54:01 bbdispproxyport = 0 2006-11-03 10:54:01 Recipient listed as '10.xxx.xxx.xxx' 2006-11-03 10:54:01 Standard BB protocol on port 1984 2006-11-03 10:54:01 Will connect to address 10.xxx.xxx.xxx port 1984 2006-11-03 10:54:01 Connect status is 0 2006-11-03 10:54:01 Sent 12 bytes 2006-11-03 10:54:01 Closing connection
bbgen --debug --report (this one turned bbgen yellow/unavailable. Note the quick disconnect.) 2006-11-03 09:51:03 load_state() 2006-11-03 09:51:03 Transport setup is: 2006-11-03 09:51:03 bbdportnumber = 1984 2006-11-03 09:51:03 bbdispproxyhost = NONE 2006-11-03 09:51:03 bbdispproxyport = 0 2006-11-03 09:51:03 Recipient listed as '10.xxx.xxx.xxx' 2006-11-03 09:51:03 Standard BB protocol on port 1984 2006-11-03 09:51:03 Will connect to address 10.xxx.xxx.xxx port 1984 2006-11-03 09:51:03 Connect status is 0 2006-11-03 09:51:03 Sent 126 bytes 2006-11-03 09:51:03 Closing connection
bbgen --debug --report (this one worked fine) 2006-11-03 09:54:00 load_state() 2006-11-03 09:54:00 Transport setup is: 2006-11-03 09:54:00 bbdportnumber = 1984 2006-11-03 09:54:00 bbdispproxyhost = NONE 2006-11-03 09:54:00 bbdispproxyport = 0 2006-11-03 09:54:00 Recipient listed as '10.xxx.xxx.xxx' 2006-11-03 09:54:00 Standard BB protocol on port 1984 2006-11-03 09:54:00 Will connect to address 10.xxx.xxx.xxx port 1984 2006-11-03 09:54:00 Connect status is 0 2006-11-03 09:54:00 Sent 126 bytes 2006-11-03 09:54:00 Read 16384 bytes 2006-11-03 09:54:00 Read 32767 bytes 2006-11-03 09:54:00 Read 1 bytes 2006-11-03 09:54:00 Read 32767 bytes 2006-11-03 09:54:00 Read 32767 bytes 2006-11-03 09:54:00 Read 24578 bytes 2006-11-03 09:54:00 Read 32767 bytes 2006-11-03 09:54:00 Read 32767 bytes 2006-11-03 09:54:00 Read 24578 bytes 2006-11-03 09:54:00 Read 16503 bytes 2006-11-03 09:54:00 Closing connection