Hi Greg,
The flapping warning is what tips it off. Flap-detection in xymond functions by looking at alternating alert states (eg, red/green) happening within a certain period and "pegging" it at the higher status while it's going back and forth. This prevents spurious recovery messages and untimely pager death.
The thing is, though, for a normal functioning 'msgs' test it's almost impossible for it to actually flap out of the box. The logfetch program which controls the raw data sent to xymond_client for evaluation actually walks back 6 "periods" (run cycles) in the log file and sends all subsequent data up to xymond. This helps with mitigating any lost messages by turning the 'msgs' test into a "Recent Errors in the Log" test instead of a direct reflection of the event, since xymon is a state-based monitoring system rather than a single-fire-and-forget (trap) based system.
Out of the box, a single red event will cause the msgs test to remain red for a solid 30m -- far too long for flapping to get triggered in most cases.
Is there any chance you have multiple servers reporting in with the name 'mgmtconsole'? Especially if you're not using FQDN (which it doesn't seem like you are), that seems like something that might cause this: Two different servers with the same name, each sending their own red/green states every few minutes.
HTH, -jc
On Thu, February 11, 2016 2:15 pm, Greg Earle wrote:
I'm running Xymon 4.3.12-2 server (yeah, I know ...) on my management system. (RHEL 6.5 currently)
A couple of days ago I migrated our central syslog server over to the Xymon server, so now "/var/log/messages" is getting a ton of stuff in it that it never had before since all my systems are now reporting in to it.
Ever since then I've seen something weird - every hour (for about 17+ hours) I was getting RED alerts for the management console's own "msgs" status, but the actual e-mail notifications don't show anything marked red in them!
It's either yellow or, as in the forwarded message below, green. I have no idea why I was getting RED alerts for this file if it thinks it's yellow or green - any ideas?
The only other thing I can add is that when I go to the Web page for mgmtconsole:msgs, it says "WARNING: Flapping status" at the top.
Is that a clue?
(Update: interestingly, it looks like the status has finally changed to green a few minutes ago - after having been red for nearly 17 1/2 hours. Still seeing "WARNING: Flapping status" on the svcstatus Web page, though.)
Thanks,
- Greg
Begin forwarded message:
From: xymon Monitor <xymon at mgmtconsole.my.do.ma.in> Subject: Xymon [750466] mgmtconsole:msgs CRITICAL (RED) Date: February 11, 2016 at 11:57:25 AM PST To: sysadmins at mgmtconsole.my.do.ma.in
green Thu Feb 11 11:57:24 PST 2016 - Log files ok <pre> </pre>
No entries in <a href="/xymon-cgi/svcstatus.sh?CLIENT=mgmtconsole&SECTION=msgs:/var/log/messages">/var/log/messages</a>
Full log <a href="/xymon-cgi/svcstatus.sh?CLIENT=mgmtconsole&SECTION=msgs:/var/log/messages">/var/log/messages</a> <...SKIPPED...> Feb 11 11:57:17 host7 nrpe[20194]: [ID 927837 daemon.info] connect from mtfuji
[... rest elided ... ]
See http://mgmtconsole/xymon-cgi/svcstatus.sh?HOST=mgmtconsole&SERVICE=msgs
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon