September 2013 - Xymon - Xymon mailman web

Admin books on Xymon
by dan＠linder.org 20 Sep '13

20 Sep '13

Does anyone have an Xymon Administrators Guide type boos (PDF or paper) they can recommend? Thanks, Dan -- ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* *****************

2 1

Two hosts, monitor a service that should be running on only one of them
by hobbit＠elyograg.org 18 Sep '13

18 Sep '13

I have a pair of machines running BBWin in central mode. Is there any way in analysis.cfg to indicate that a particular service exists on both of them but should only be running on one of them? If that service is running on both servers or neither server, I want to see a red alarm. Thanks, Shawn

3 3

graph shows incorrect data
by naveen20＠gmail.com 18 Sep '13

18 Sep '13

Hi, I am new to Xymon. I have setup Xymon to monitor my performance servers. I have written custom scripts to collect some specific data, and added it to clientlaunch.cfg to run at every 5 second interval [gc] ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg CMD $XYMONCLIENTHOME/ext/gc_stats.sh LOGFILE $XYMONCLIENTHOME/tmp/msg.gc_stats.log INTERVAL 5 gc_stats.sh looks like this. ====================== pid=`pgrep rdu` #Fact=1024.0 Fact=0.0009765 gc=`/opt/jdk1.6.0_24/bin/jstat -gc ${pid} |tail -1` IFS=' ' read -a array <<< "$gc" EdenCap=${array[4]} EdenUsed=${array[5]} OldCap=${array[6]} OldUsed=${array[7]} PermCap=${array[8]} PermUsed=${array[9]} EdenCap=`echo "$EdenCap * $Fact"|bc` EdenUsed=`echo "$EdenUsed * $Fact"|bc` OldCap=`echo "$OldCap * $Fact"|bc` OldUsed=`echo "$OldUsed * $Fact"|bc` PermCap=`echo "$PermCap * $Fact"|bc` PermUsed=`echo "$PermUsed * $Fact"|bc` COLOR=green COLUMN=HeapStats MSG="EdenCapacity: ${EdenCap} EdenUsed: ${EdenUsed} OldCapacity: ${OldCap} OldUsed: ${OldUsed} PermCapacity: ${PermCap} PermUsed: ${PermUsed} " # Tell Xymon about it $XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR `date` ${MSG} " exit 0 ======================================== On the server side, I have added the following in xymonserver.cfg ------------------------- TEST2RRD="cpu=la,disk,inode,qtree,memory,$PINGCOLUMN=tcp,http=tcp,dns=tcp,dig=tcp,time=ntpstat,vmstat,iostat,netstat,temperature,apache,bind,sendmail,mailq,nmailq=mailq,socks,bea,iishealth,citrix,bbgen,bbtest,bbproxy,hobbitd,files,procs=processes,ports,clock,lines,ops,stats,cifs,JVM,JMS,HitCache,Session,JDBCConn,ExecQueue,JTA,TblSpace,RollBack,MemReq,InvObj,snapmirr,snaplist,snapshot,if_load=devmon,temp=devmon,paging,mdc,mdchitpct,cics,dsa,getvis,maxuser,nparts,xymongen,xymonnet,xymonproxy,xymond,HeapStats=ncv" NCV_HeapStats="EdenCapacity:GAUGE,EdenUsed:GAUGE,OldCapacity:GAUGE,OldUsed:GAUGE,PermCapacity:GAUGE,PermUsed:GAUGE" GRAPHS="la,disk,inode,qtree,files,processes,memory,users,vmstat,iostat,tcp.http,tcp,ncv,netstat,ifstat,mrtg::1,ports,temperature,ntpstat,apache,bind,sendmail,mailq,socks,bea,iishealth,citrix,bbgen,bbtest,bbproxy,hobbitd,clock,lines,ops,stats,cifs,JVM,JMS,HitCache,Session,JDBCConn,ExecQueue,JTA,TblSpace,RollBack,MemReq,InvObj,snapmirr,snaplist,snapshot,devmon::1,if_load::1,temp,paging,mdc,mdchitpct,cics,dsa,getvis,maxuser,nparts,xymongen,xymonnet,xymonproxy,xymond,HeapStats" ---------------------------- I have added the below in graphs.cfg ============================== [HeapStats] TITLE HeapStats info YAXIS MegaBytes -l 0.000 DEF:EdenUsed=HeapStats.rrd:EdenUsed:AVERAGE DEF:EdenCapacity=HeapStats.rrd:EdenCapacity:AVERAGE DEF:OldCapacity=HeapStats.rrd:OldCapacity:AVERAGE DEF:OldUsed=HeapStats.rrd:OldUsed:AVERAGE DEF:PermCapacity=HeapStats.rrd:PermCapacity:AVERAGE DEF:PermUsed=HeapStats.rrd:PermUsed:AVERAGE LINE1:EdenCapacity#FF0000: EdenCapacity LINE2:EdenUsed#808080: EdenUsed LINE3:OldCapacity#00CC00: OldCapacity LINE4:OldUsed#00CCCC: OldUsed LINE5:PermCapacity#FFFF00: PermCapacity LINE6:PermUsed#CFFFF0: PermUsed COMMENT:\n GPRINT:EdenCapacity:LAST:EdenCapacity \: %5.1lf (cur) GPRINT:EdenCapacity:MAX: \: %5.1lf (max) GPRINT:EdenCapacity:MIN: \: %5.1lf (min) GPRINT:EdenCapacity:AVERAGE: \: %5.1lf (avg)\n COMMENT:\n GPRINT:EdenUsed:LAST:EdenUsed \: %5.1lf (cur) GPRINT:EdenUsed:MAX: \: %5.1lf (max) GPRINT:EdenUsed:MIN: \: %5.1lf (min) GPRINT:EdenUsed:AVERAGE: \: %5.1lf (avg)\n COMMENT:\n GPRINT:OldCapacity:LAST:OldCapacity \: %5.1lf (cur) GPRINT:OldCapacity:MAX: \: %5.1lf (max) GPRINT:OldCapacity:MIN: \: %5.1lf (min) GPRINT:OldCapacity:AVERAGE: \: %5.1lf (avg)\n COMMENT:\n GPRINT:OldUsed:LAST:OldUsed \: %5.1lf (cur) GPRINT:OldUsed:MAX: \: %5.1lf (max) GPRINT:OldUsed:MIN: \: %5.1lf (min) GPRINT:OldUsed:AVERAGE: \: %5.1lf (avg)\n COMMENT:\n GPRINT:PermCapacity:LAST:PermCapacity \: %5.1lf (cur) GPRINT:PermCapacity:MAX: \: %5.1lf (max) GPRINT:PermCapacity:MIN: \: %5.1lf (min) GPRINT:PermCapacity:AVERAGE: \: %5.1lf (avg)\n COMMENT:\n GPRINT:PermUsed:LAST:PermUsed \: %5.1lf (cur) GPRINT:PermUsed:MAX: \: %5.1lf (max) GPRINT:PermUsed:MIN: \: %5.1lf (min) GPRINT:PermUsed:AVERAGE: \: %5.1lf (avg)\n ============================ In the HeapStats (column) page, I see the latest values obtained in the text message like this. ========= Tue Sep 10 12:15:48 IST 2013 EdenCapacity: 1080.8058240 EdenUsed: 1053.1013472 OldCapacity: 2744.8243200 OldUsed: 91.4092861 PermCapacity: 44.9346240 PermUsed: 44.7480148 ============ But, when it comes to graph it is showing stale data always. It is not reflecting the latest data [image: xymongraph ncv:HeapStats] ====== Sorry for the long mail. Could some one help me with this? I want all the data being collected to be persisted in the rrd files. Any help is very much appreciated. Thanks, NV

2 2

xymon 4.3.12: ntp
by hj.beckers＠kreis-steinfurt.de 17 Sep '13

17 Sep '13

Since upgrading, my ntp test doesn't work anymore. OS is openSuSE 11.3. It seems I cannot set any parameters in xymonserver.cfg. OpenSuSE seems not to recognize the "-p 2" ? Command: /usr/sbin/sntp -u -q -p 2 IP.adress 2>&1 Syntax: sntp [ --help | -h | -? ] [ -v | -V | -W ] [ -q [ -f savefile ] | [ [ -F ] { -r | -a } [ -P prompt ] [ -l lockfile ] ] [ -c count ] [ -e minerr ] [ -E maxerr ] [ -d delay | -x [ separation ] [ -f savefile ] ] [ -4 | -6 ] [-u] [ address(es) ] ] Hermann-Josef Beckers

2 1

http tests failing some of the time
by david.gore＠verizon.com 15 Sep '13

15 Sep '13

I have some http tests that are green most of the time but randomly, typically less than 5 minutes elapse, before they timeout. I have tried all sorts of things to no avail. The http test when green should take no more than a second or two to return but is instead taking 9-14 seconds. 1) -no-ares option 2) xymonnet --debug option, I see where it times out in the debug output but it isn't helpful 3) changed dns server order in /etc/resolv.conf 4) installed nscd 5) changed https to https3, although some of the tests are http and some are https 6) added the ip to the http line in hosts.cfg 7) the http test for the xymon server itself timesout but other remote sites are always green 8) added --concurrency=50 and 100, no help Works perfectly on a xymon server running RedHat 5.9 32-bit but times out for a small fraction of the http hosts on RHEL 6.4 64-bit. Very odd. Any clues, tips and troubleshooting welcome. Thanks in advance. ~David

1 0

xymond_locator documentation and other thoughts
by sean.clark＠twcable.com 12 Sep '13

12 Sep '13

xymond_locator program - I've read through the code and it makes sense but is there (going to be?) a man page or further explanation ? I wrote a history & reflector style module that listens to channels - this looks like a more formal way to do it? Is it possible to have multiple display destinations running on locator? ps -- use "restofmsg" NOT metadata[15] -- because Henrik might just toss 4 more new metadata things in there and throw your whole code off :-D pps - Japeth, Henrik, Milne, (and I guess Sean who is the grandaddy of all of this) - thanks for all your hard work in making such a great enterprise program! I really must thank Japeth Cleaver again for his patches and rpm structure (and selinux option). Also a big thanks to the community here and any one who has contributed that I failed to mention. ppps - Japeth - the history one isn't the piece of junk I was talking to you about 3 years back or so - I actually learned how to write much better C and how it works without any issues other than the number of metadata elements changing pppps (last one) - new-daemon.txt is the only place where the channels were formally explained, and that file is woefully out of date - I would suggest either dropping the file or updating it with the extra fields in it -- Sean Clark This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

2 1

Possible omission in the ~/client/etc/xymonclient.cfg file?
by martin＠savcom.co.uk 12 Sep '13

12 Sep '13

Hi. >From the documentation around the xymonclient.cfg config file it is supposed to set up the environment for client-side scripts so that they can get hold of environment variables such as XYMON and XYMSRV (BB & BBDISP in Hobbit days) I've just moved from Hobbit to Xymon and I am getting errors on the results of my scripts because the environment variables I used in Hobbit are not defined in Xymon. I think the reason is as follows: In xymonclient.cfg under Compatibility Settings, BB is defined here: BB="$XYMON" Earlier on in the file XYMON is defined here: XYMON="$XYMONHOME/bin/xymon" # The Xymon client "xymon" utility And looking further back XYMONHOME is defined here: XYMONHOME="$XYMONCLIENTHOME" # Directory for the Xymon client files But XYMONCLIENTHOME is not defined anywhere I can find in xymonclient.cfg, which means ultimately BB is not defined. Therefore I get the following from my Perl scripts: Use of uninitialized value $ENV{"BB"} in concatenation (.) or string at ./CurlFile.pl line 144. Use of uninitialized value in concatenation (.) or string at ./CurlFile.pl line 144. sh: 8: status Internet.Perf green Sat Sep 7 14:15:16 BST 2013 - OK This string breaks the logic of the BB messaging and results in a RED alert on my monitoring. I know that XYMONCLIENTHOME is defined in xymonserver.cfg as XYMONCLIENTHOME="/home/xymon/client" Am I right that I should define XYMONCLIENTHOME somewhere in xymonclient.cfg and that it should be the same as XYMONCLIENTHOME="/home/xymon/client" or is some other logic broken between xymonserver.cfg and xymonclient.cfg? Cheers Martin

2 1

is depends broken?
by troy＠athabascau.ca 12 Sep '13

12 Sep '13

So I did a little testing with "route:". The behavior I see when I test is (where "child" is the machine depended on the "parent"): By default (no route directive used here), when the child:conn and the child:http both go red then the child:http is turned clear because of innate dependency on conn. With the route directive in place, when the parent:conn, child:conn and the child:http all go red then the child:http is turned clear (same as above) but the child:conn bounces between yellow and red (with an approximate cycle of 1:30 red and 30s yellow). Next when I bring up the parent:conn (restore ping function), the child:conn goes red and stays red. I guess the bottom line is, I can't get "route" or "depends" to do what I want or even what the manual says they do. Looks to me like something is broken real bad. I am using Xymon 4.3.10. Anybody have a clue what to do? cheers, Troy ----- Original Message ----- From: "Troy Adams" <troy at athabascau.ca> To: "Josh Luthman" <josh at imaginenetworksllc.com> Cc: xymon at xymon.com Sent: Thursday, August 22, 2013 11:26:22 AM GMT -07:00 US/Canada Mountain Subject: Re: [Xymon] is depends broken? Galen, I should have also stated that google was up for the entire period (no flapping). Josh, I take it, you mean replace my previous configuration where it said: depends=(http: google.com/http ) and make it: route:google.com so it ends up as: XXX.XXX.XXX.XXX www # trace http://www route:google.com I tried that and when I block both hosts google/icmp,http and www:http with iptables they both show red, I was expecting www:http to show clear to get the same functionality as "depends". Any thoughts? cheers, Troy ----- Original Message ----- From: "Josh Luthman" <josh at imaginenetworksllc.com> To: "Galen Johnson" <Galen.Johnson at sas.com> Cc: xymon at xymon.com, "Troy Adams" <troy at athabascau.ca> Sent: Monday, August 19, 2013 8:56:57 PM GMT -07:00 US/Canada Mountain Subject: Re: [Xymon] is depends broken? Use route: other.host.com Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Aug 19, 2013 10:31 PM, "Galen Johnson" < Galen.Johnson at sas.com > wrote: #2 sounds a lot like "flapping". Does "depends" honor flapping or would a flapping server still show red? Or am I not understanding the situation? =G= From: Xymon [ xymon-bounces at xymon.com ] on behalf of Troy Adams [ troy at athabascau.ca ] Sent: Monday, August 19, 2013 5:34 PM To: xymon Subject: [Xymon] is depends broken? I have the following in my 'hosts.cfg': 209.85.225.105 google.com # http://www.google.com XXX.XXX.XXX.XXX www # trace http://www depends=(http: google.com/http ) Yesterday, the web server (www) was down for 55 minutes. The Xymon record shows that: 1. google.com/http was green for that whole period 2. www/http bounced between red and clear for the whole period (with a bounce frequency of 5 minutes) 3. My other Xymon server (configured without "depends") shows a solid 55 minute outage (red for 55 minutes). This behavior is not what I expect from reading the man page. I tried to reproduce this problem with "depends" on my test Xymon server and was able to confirm the same behavior. Is there a workaround to this problem with my Xymon 4.3.10 server? cheers, Troy This communication is intended for the use of the recipient to whom it is addressed, and may contain confidential, personal, and or privileged information. Please contact us immediately if you are not the intended recipient of this communication, and do not copy, distribute, or take action relying on it. Any communications received in error, or subsequent reply, should be deleted or destroyed. _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

2 1

--pagetitle-links affected by HOSTDOCURL
by john.thurston＠alaska.gov 11 Sep '13

11 Sep '13

It appears that linking to "notes" files takes precedence over linking to pages. Is there a way to override this behavior? My example: Modify xymonserver.cfg: add --pagetitle-links to XYMONGENOPTS edit HOSTDOCURL to point to something like foo.php?host=%s The result: Every host name is now linked to foo.php?host=hostname Every page name is now linked to foo.php?host=pagename Every page "bubble" is still linked to /xymon/page/page.html If the value for HOSTDOCURL is commented out again, the page names are back to links to the pages. Is there something like a NOPAGENOTES option which will stop xymongen from using HOSTDOCURL to create links for page names? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

1 0

RECOVERED notification
by bb4＠richter-it.net 09 Sep '13

09 Sep '13

Hi folks, I am running XYmon 4.3.12 (self compiled) on a RHEL 6.3 and have a question about the behaviour of sending "RECOVERED" messages. I've been forced to send SNMP traps of alerts to a Patrol console and of course they also want to have "RECOVERED" messages to close these alerts automatically. But now I've seen in my logs that if someone disables a test that has an alert another alert trap is being sent out instead of a "RECOVERED" trap. I.e. I have a server that has an alert in msgs an red alert is sent to Patrol and the responsible admin is called. He sets the state of msgs to blue for a certain time and another red alert is sent to Patrol instead of a green trap saying everything back to normal. Is this an expected behaviour? I thought even blue would trigger a recovery message since I have the following lines in my xymonserver.cfg: ALERTCOLORS="red,yellow,purple" # Colors that may trigger an alert message OKCOLORS="green,blue,clear" # Colors that may trigger a recovery message Any help would be appreciated. Regards Torsten

1 1