August 2018 - Xymon - Xymon mailman web

Setting thresholds in analysis.cfg
by jeremy＠laidman.org 21 Aug '18

21 Aug '18

Chris I think this is the key part of the man page: HOST=targetstring Rule matching a host by the hostname. "targetstring" is either a comma-separated list of hostnames (from the hosts.cfg file), "*" to indicate "all hosts", or a Perl-compatible regular expression. Are your host definitions comma-separated lists, or PCREs? They can't be both. So none of your hosts match, and the DEFAULT stanza is the one that applies. J On 9 June 2018 at 02:43, Seip, Christopher (HPN SIS team) < chris.seip at hpe.com> wrote: > I could a hand getting the basics of analysis.cfg worked out, please. > Here's mine: > > # egrep -v '^#' analysis.cfg > > HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 > DISK /disk/data 50 55 > DISK * 90 95 > MEMSWAP 80 90 > > HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 > DISK /disk/data 92 96 > DISK * 90 95 > > HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 > DISK /BACKUP 98 99 > DISK * 90 95 > > DEFAULT > # Ignore some usually uninteresting tmpfs mounts. > DISK /dev IGNORE > DISK /dev/shm IGNORE > DISK /lib/init/rw IGNORE > DISK /run IGNORE > # These are the built-in defaults. You should only modify these > # lines, not add new ones (no PROC, DISK, LOG ... lines). > UP 1h > LOAD 5.0 10.0 > DISK * 90 95 > INODE * 70 90 > MEMPHYS 100 101 > MEMSWAP 50 80 > MEMACT 90 97 > > > Three issues with this: > > 1. Swap consumption in the first host, swnfs06, has been steady at 74%, so > I was trying to hush the alerts with the MEMSWAP line. This change hasn't > had any effect; I am still getting a memory low yellow-warning for > swap/page usage on swnfs06. > > 2. On the same swnfs06 host, its /disk/data partition is 56% full, so my > "DISK..50 60" line was an attempt to trigger a yellow alert. I was testing > my understanding of the analysis.cfg file, but the filesystems test remains > green. > > 3. And my 96% full /BACKUP drive on hpnsvr18 is issuing a red alert for > being over the panic level of 95%, where I was trying to set the panic > level at 99%. > > After wrestling with the man page and many experiments, I'm tossing this > to the group for help. Seems very basic, but it's just not working for me. > What'm I missing? > > I tried switching to the "threshold hostname" format, like this: > > # egrep -v '^#' analysis.cfg | head -11 > > DISK /disk/data 50 55 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 > DISK * 90 95 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 > MEMSWAP 80 90 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 > > DISK /disk/data 92 96 HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 > DISK * 90 95 HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 > > DISK /BACKUP 98 99 HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 > DISK * 90 95 HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 > > This produced no change in behavior. I am stopping and starting the Xymon > server software and waiting for new html pages to generate after every > change in the analysis.cfg. > > In my configuration report, I can see that every server configured for > local memory tests has acquired the 80%/90% threshold setting, not just > swnfs06. And my "DISK /disk/data 50 55" is having no effect at all on any > host: The strings "50%" or "60%" appear nowhere in my configuration report. > > # egrep 'swnfs0[67]' hosts.cfg > 16.93.247.204 swnfs06.rose.rdlabs.hpecorp.net # > rpc=mount,nlockmgr,nfs,ypbind ssh > 16.93.247.205 swnfs07.rose.rdlabs.hpecorp.net # NOCOLUMNS=files > rpc=mount,nlockmgr,nfs,ypbind ssh > 16.93.247.204 swnfs06.rose.rdlabs.hpecorp.net > > # xymoncmd xymond_client --dump-config > DISK /disk/data 50% 55% 0 -1 red HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 > (line: 351) > DISK * 90% 95% 0 -1 red HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 > (line: 352) > MEMSWAP 80 90 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 (line: 353) > DISK /disk/data 92% 96% 0 -1 red HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 > (line: 355) > DISK * 90% 95% 0 -1 red HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 > (line: 356) > DISK /BACKUP 98% 99% 0 -1 red HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 > (line: 360) > DISK * 90% 95% 0 -1 red HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 > (line: 361) > DISK /dev IGNORE (line: 371) > DISK /dev/shm IGNORE (line: 372) > DISK /lib/init/rw IGNORE (line: 373) > DISK /run IGNORE (line: 374) > UP 3600 -1 (line: 377) > LOAD 5.00 10.00 (line: 378) > DISK * 90% 95% 0 -1 red (line: 379) > INODE * 70% 90% 0 -1 red (line: 380) > MEMREAL 100 101 (line: 381) > MEMSWAP 50 80 (line: 382) > MEMACT 90 97 (line: 383) > > Thanks for any insights you can provide. Feels like I'm making a wrong > assumption about how analysis.cfg works. > > Best thing I can figure to do would be to switch to local configuration of > my Xymon clients, but I'd rather manage custom thresholds centrally. > > - Chris > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

2 1

Ghost-busting... how to locate a ghost client
by glauber.ribeiro＠experian.com 21 Aug '18

21 Aug '18

And it worked! Thank you. g From: Ribeiro, Glauber Sent: Tuesday, August 21, 2018 08:18 To: Root, Paul T <Paul.Root at CenturyLink.com> Subject: RE: Ghost-busting... how to locate a ghost client That is an excellent idea, thanks! g From: Root, Paul T [mailto:Paul.Root at CenturyLink.com] Sent: Monday, August 20, 2018 16:58 To: Ribeiro, Glauber <glauber.ribeiro at experian.com<mailto:glauber.ribeiro at experian.com>>; xymon >> xymon at xymon.com<mailto:xymon at xymon.com> <xymon at xymon.com<mailto:xymon at xymon.com>> Subject: RE: Ghost-busting... how to locate a ghost client So that would be the server is performing a test on the non-existent machine. Look in tasks.cfg or tasks.d. Or maybe it’s a crontab. The other thing you can do is put the non-existent machine into hosts.cfg and then eventually you will get the test to show up, so you know what it is that is running on that machine. From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Ribeiro, Glauber Sent: Monday, August 20, 2018 4:28 PM To: xymon >> xymon at xymon.com<mailto:xymon at xymon.com> <xymon at xymon.com<mailto:xymon at xymon.com>> Subject: [Xymon] Ghost-busting... how to locate a ghost client I have a non-existent server which shows constantly in my “Ghost clients” report. I think there must be a test somewhere that’s reporting the wrong host name. Is there anything I can do to find out where that message is coming from? The Ghost clients report tells me the sender is “127.0.0.1” (of course). Thanks, glauber -- glauber.ribeiro at experian.com<mailto:glauber.ribeiro at experian.com> This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.

1 0

xymon patches
by xymon＠nndev.org 21 Aug '18

21 Aug '18

Is there a proper place to send patches? I have a couple: One fixes xymon so it compiles on Fedora 28, plus some type mismatch cleanup The other fixes do_temperature.c so it actually works correctly I also have a re-written temperature.sh script that works on Solaris, FreeBSD, and Linux. I can stick them on my website, but I'd rather get them actually incorporated into the codebase.

2 1

generate eventlog report from shell ?
by bakgat8＠hotmail.com 21 Aug '18

21 Aug '18

Is there a way to generate an event log report from the shell? I want to email a monthly report of change in status for column X . One of those things where one failure is not alert-worthy, but if the problem keep re-occurring, somebody need to look. Bakkies

2 1

Ghost-busting... how to locate a ghost client
by glauber.ribeiro＠experian.com 20 Aug '18

20 Aug '18

I have a non-existent server which shows constantly in my “Ghost clients” report. I think there must be a test somewhere that’s reporting the wrong host name. Is there anything I can do to find out where that message is coming from? The Ghost clients report tells me the sender is “127.0.0.1” (of course). Thanks, glauber -- glauber.ribeiro at experian.com

2 1

NTP requirement for Terabithia xymon RPM - RHEL/CentOS 7?
by mburger＠bubbanfriends.org 18 Aug '18

18 Aug '18

Good morning, I'm installing Xymon on an Amazon Linux 2 instance (RHEL/CentOS 7 equivalent) and noticed that, although Chrony is installed on the instance, the ntp and ntpdate packages were installed as pre-reqs for the JC's xymon package. I was just wondering: * Why is ntp/ntpdate a requirement for the Xymon package? * Does Chrony not have equivalent functionality? Thanks. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1

2 1

monitoring contents of a logfile with a daily changing filename
by EDSchminke＠Hormel.com 17 Aug '18

17 Aug '18

Hang on. Thats not entirely correct. Xymon does not look at "the last SIZE bytes of the log file". Through a coincidence, it might.. but that's what it does. The rules that govern what gets returned is a little more complicated, but important to understand to avoid tearing out all your hair while troubleshooting. The SIZE component of the LOG entry only specifies the maximum amount of data to send back to the server. The logfetch program on the client side will take the last 30 minutes (kinda) of the file into consideration for what it sends back. An IGNORE rule removes lines from consideration (will not be sent, will not count against the max SIZE). Then, TRIGGER rules will send all matched lines even if it exceeds max SIZE. If what was found by any TRIGGER rules is less than max SIZE, it will include the difference from any remaining lines, up to the max SIZE. Still, only the last 30 minutes (kinda) are considered. I say kinda, because the logfetch program works like this. Every time the logfetch program checks a log file, it takes note of the current size of the log file. It keeps track of this in the STATUSFILE. (See logfetch manpage). Each line of the STATUSFILE lists the log files it's watching followed by a "queue" of numbers. Those numbers represent the size of the log file at the last 6 times it was checked. Every time logfetch runs, it unshifts the current size of the log file onto the front of the queue and pops the last number off the end of the queue. Then, logfetch opens the log file, seeks to the byte number that it popped off the queue, and reads to the end of file. So, log fetch returns the last "6 * check interval" minutes worth of entries in the log. Check interval is USUALLY 5 minutes, hence the 30 minutes. If it's not returning what you're expecting to get back from the logs, it's most likely due to how logfetch only concerns itself with that "last 6 checks" worth of the log. Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com

2 1

analysis.cfg - match a sentence not a single word
by mburger＠bubbanfriends.org 16 Aug '18

16 Aug '18

In my implementation, we've got % at the front of our text regex: LOG %/var/log/messages.*.log "%Failed to initialize credentials using keytab" COLOR=yellow You might give that a try. On 2018-08-16 11:53, Ian Diddams via Xymon wrote: > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1

2 1

analysis.cfg - match a sentence not a single word
by didds3＠yahoo.co.uk 16 Aug '18

16 Aug '18

Follwoing on from my rpevious queries and thanks to all that helped, Ive now got everything working but matching a single word ("DIDDS"). however - what I actually need to match is a sentence eg "Failed to initialize credentials using keyta" I can try this LOG %/var/log/messages.*.log "Failed to initialize credentials using keytab" COLOR=yellow but the GUI flags the contents of the file as containing that but it doesn;t alert for it. all the man page suggests is single word pattern matching is available - any other suggestions? cheers

1 0

monitoring contents of a logfile with a daily changing filename
by didds3＠yahoo.co.uk 16 Aug '18

16 Aug '18

I realised the solution came about via direct emails with Andy so here is the solution for posterity: Now the LOG config in analysis.cfg is simply :LOG %/var/log/messages.*.log "XXXX" COLOR=yellow And the accompanying client-local.cfg entry is :[linux]log:/var/log/messages:10240log:`find /var/log -maxdepth 1 -type f -name messages-\*.log`:10240000000 The requirement was the regex wildcar .* not the glob of just *. Next question... I now see if I place the trigger word DIDDS in an appropriate logfile xymon does spot it and alert on it, and the GUI page shows the contents of the logfile as <CURRENT> DIDDS But after 10 minutes... it drops the alert and the gui page no longer also lists those contents - in fact it shows nothing as the contents. Anybody any ideas as to why this behaviour is seen. These tests are at the moment using dummy logfilesd which have no other info in them. It is literally only the line "DIDDS" which remains there. If I add another line DIDDS once again it spots it and then alerts and shows it as <CURRENT> DIDDS. For ten minutes - then it all drops again... any ideas why this is occurring? didds

1 0