I installed xynagios, which allows dropin execution of nagios plugins in xymon (and a variant xynagios.yellow ) Since there are a lot of useful nagios plugins out there, this represents a huge increase in tests we can use off-the-shelf.
Using nagios plugins has an interesting ‘gotcha’. Nagios is event driven ; if it receives an alert, that alert stays unchanged until either a human clears it or a new alert arrives. Xymon however is poll-driven; it expects alert statuses at regular intervals and turns an alert purple if no updates occur within the lifetime (set clientside by the “status” variable in the returned line). As xynagios is run regularly, it does the right thing to update single alerts. But what happens if a Nagios test sends more than one sort of alert?
I installed the check_hp_bladechassis plugin, which uses snmp. It sends an “hp_bladechassis” alert when snmp is up but an “snmp” alert when snmp fails. So how to keep these from going stale?
What I did was use the ‘xymondboard’ feature to query board status (perl code below)
If snmp has not logged recently, we know it must no longer be yellow (since xynagios would have sent a fresh yellow) so set it to green If snmp is yellow, we know the hp_bladechassis data is stale, so set it to clear.
my $boardinfo =
$BB $BBDISP "xymondboard host=$blade test=snmp fields=color,logtime";
( $color, $logtime ) = split( '\|', $boardinfo );
if ( ( $epoch - $logtime ) > $STALE ) {
$bbcmd =
"$BB $BBDISP 'status+2h $blade.snmp green $date SNMP now OK on $blade'";
system($bbcmd);
}
if ( $color eq "yellow" ) {
$bbcmd ="$BB $BBDISP 'status+2h $blade.hp_bladechassis clear $date SNMP error unable to run HP test on $blade'"; system($bbcmd); } }