I think I'm getting bogus, occasional pages from an UNMATCHED rule that should not be matching. (xymon 4.3.10)
I set up a default UNMATCHED rule as the last rule in alerts.cfg: (just recently added the DURATION)
HOST=* SERVICE=* SCRIPT /usr/local/scripts/mailunmatched betsy UNMATCHED DURATION>10 FORMAT=TEXT
However, painfully, I am getting pages *occasionally* and incorrectly.
They all seem to be coming from this rule set:
HOST=* EXHOST=%($NOPAGE|$VIP) SERVICE=$TECHOPS_TESTS include techops.escalation.cfg
where techops.escalation.cfg is MAIL techopsmail repeat=8h COLOR=yellow FORMAT=TEXT MAIL techopsmail repeat=5m COLOR=red FORMAT=TEXT MAIL alert1 REPEAT=10 COLOR=red FORMAT=SMS MAIL alert2 DURATION>20 REPEAT=10 COLOR=red FORMAT=SMS MAIL alert3 DURATION>40 REPEAT=10 COLOR=red FORMAT=SMS MAIL alert4 DURATION>60 REPEAT=10 COLOR=red FORMAT=SMS MAIL alert5 DURATION>80 REPEAT=10 COLOR=red FORMAT=SMS
It's only happening *occasionally*. When I test the rules, ie /usr/local/xymon/server/bin/xymond_alert --test bm1-08.example.com disk --color=yellow they don't hit the unmatched rule.
For example, today, bm1-08 which hits the above rules has had a busy disk day:
Thu Dec 13 20:22:25 2012 bm1-08.example.com disk green From -> To yellow Thu Dec 13 15:50:00 2012 bm1-08.example.com disk yellow From -> To green Thu Dec 13 13:48:54 2012 bm1-08.example.com disk green From -> To yellow Thu Dec 13 13:28:44 2012 bm1-08.example.com disk yellow From -> To green Thu Dec 13 13:13:45 2012 bm1-08.example.com disk red From -> To yellow Thu Dec 13 12:58:45 2012 bm1-08.example.com disk yellow From -> To red Thu Dec 13 12:48:44 2012 bm1-08.example.com disk green From -> To yellow Thu Dec 13 10:28:11 2012 bm1-08.example.com disk yellow From -> To green Thu Dec 13 10:18:06 2012 bm1-08.example.com disk green From -> To yellow Thu Dec 13 09:57:54 2012 bm1-08.example.com disk yellow From -> To green Thu Dec 13 09:42:46 2012 bm1-08.example.com disk green From -> To yellow Thu Dec 13 08:22:12 2012 bm1-08.example.com disk yellow From -> To green Thu Dec 13 07:36:56 2012 bm1-08.example.com disk green From -> To yellow Thu Dec 13 05:41:37 2012 bm1-08.example.com disk yellow From -> To green Thu Dec 13 05:21:35 2012 bm1-08.example.com disk green From -> To yellow
and I was paged, erroneously, **ONCE**
Thu Dec 13 05:22:36 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355394156 100 Thu Dec 13 07:37:23 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355402243 100 Thu Dec 13 09:42:46 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355409766 100 Thu Dec 13 10:18:35 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355411915 100 Thu Dec 13 12:48:44 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355420924 100 Thu Dec 13 12:58:45 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[253] 1355421525 100 Thu Dec 13 12:58:45 2012 bm1-08.example.com.disk (10.101.251.108) alert1[254] 1355421525 100 Thu Dec 13 13:03:45 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[253] 1355421825 100 Thu Dec 13 13:48:55 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355424534 100 Thu Dec 13 20:22:25 2012 bm1-08.example.com.disk (10.101.251.108) techopsmail[252] 1355448145 100 Thu Dec 13 20:32:32 2012 bm1-08.example.com.disk (10.101.251.108) betsy 1355448752 100
This feels like a bug. Anyone else experiencing anything like this? Or am I doing something wrong?