September 2015 - Xymon - Xymon mailman web

Ext to alert once a day
by rltexas1＠gmail.com 30 Sep '15

30 Sep '15

I have an ext script that runs once every 24 hours. How can I configure the alerts to only send an email once a day if the status is x color. Thanks

2 2

BUG: ntpdate command used in Xymon takes 2+ seconds each in RHEL 7 (with ntp 4.2.6+)
by mvanderw＠nd.edu 30 Sep '15

30 Sep '15

Hello J.C., etc. (Brief Summary) I noticed a issue/bug in the way Xymon uses the ntpdate command for it's built-in ntp checks. Each ntp test (using the ntpdate options hard-coded into Xymon) takes >= 2 seconds to run in any OS that uses upstream ntp version 4.2.6 or later, including RHEL 7. This is due to a 2 second delay that was put in place in upstream ntp starting with version 4.2.6. While RHEL 6 uses upstream ntp 4.2.6 in it's latest iteration, the behavior I'm referring to was reverted by Red Hat when RHEL 6 moved from using version 4.2.4 to 4.2.6 in a minor release. This was because this kind of major change in behavior was not desired in only a minor RHEL release. However, in RHEL 7 it was acceptable to keep this behavior change in place since it was a major OS release. (Detailed (continued) Summary) Initially, I thought it was a Red Hat issue, so a support ticket was opened up with Red Hat to figure out what the issue was. The support person was able to replicate the issue I was seeing and created a public Bugzilla bug here: https://bugzilla.redhat.com/show_bug.cgi?id=1260140. The ntp package maintainer for Red Hat replied saying this was intended behavior and while reverted in RHEL 6, it would not be reverted in RHEL 7. I asked in the support ticket why it was not going to be fixed in RHEL 7 and got this response in the support ticket: "The original ntp version shipped in RHEL6 was 4.2.4, which had ntpdate that was fast, but violated the default 2s minimum spacing between requests. An NTP server with enabled rate limiting would not respond more than once to such client. In ntp 4.2.6 that bug was fixed by adding spacing between the requests, which slows down the ntpdate operation. This change in behavior was not acceptable for a minor RHEL6 release, so the bug was restored in a patch. In RHEL7 as a new major release I think such change is acceptable and ntpdate now works as upstream intended." Please see public Bugzilla bug for additional links for reference of this change in behavior. (Request) *It would be greatly appreciated if this could be fixed in upstream Xymon ASAP! However, since my Xymon server was installed using Terabithia RPMs, what is most important to me is to get an updated Terabithia RPM created that I can use to update my Xymon server. I understand that a source code patch has already been created for Debian Xymon here: https://anonscm.debian.org/cgit/collab-maint/xymon.git/tree/debian/patches/… (referenced in earlier mailing list post: http://lists.xymon.com/pipermail/xymon/2015-September/042224.html) This would work perfectly!! However, since I am using Terabithia RPMs, it is not very easy to simply apply this above patch and recompile the base source code and use that, since I am I not using just the base source code. Maybe a hotfix release of Xymon could be created and released sometime very soon with this fix? Currently, I am unable to do all the ntp tests I would like to do because it causes xymonnet to go over the 5 minute interval time!.... In the meantime, would it be possible for someone (maybe J.C.) to create a binary patch or something that I could apply to the current xymonnet binary with the fix in the above patch? Just a thought...and would be extremely appreciated!! Thank you very much for your time and work you do to make Xymon a great monitoring tool! I look forward to some fixes being applied to resolve this issue with the ntp tests! TL;DR: ntp checks take WAY to long in Xymon using hard-coded ntpdate options, due to change in ntpdate behavior starting in upstream ntp version 4.2.6. This particularly affects Xymon servers running on RHEL 7 (and new releases in the future). Hotfix needed in Xymon to fix this issue (see source code patch above) ASAP! If possible, a binary patch to xymonnet binary would be much preferred in the meantime! I am using Terabithia RPMs for my Xymon server, so using above source code patch is not possible or desirable! FYI: I am running Xymon 4.3.21-4.el7.terabithia for my Xymon server. Thanks!! -- Matt Vander Werf

2 3

buffer overflow detected in xymongen (4.3.21)
by beckert＠phys.ethz.ch 30 Sep '15

30 Sep '15

Hi, today our xymongen check went purple for about an hour. In the xymongen.log I found tons of crash reports like this one: *** buffer overflow detected ***: xymongen terminated ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x731ff)[0x7f31a13fc1ff] /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f31a147f4c7] /lib/x86_64-linux-gnu/libc.so.6(+0xf46e0)[0x7f31a147d6e0] xymongen[0x40d526] xymongen[0x403b72] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f31a13aab45] xymongen[0x40510c] ======= Memory map: ======== 00400000-00445000 r-xp 00000000 fd:00 151583 /usr/lib/xymon/server/bin/xymongen 00644000-00645000 r--p 00044000 fd:00 151583 /usr/lib/xymon/server/bin/xymongen 00645000-00647000 rw-p 00045000 fd:00 151583 /usr/lib/xymon/server/bin/xymongen 00647000-00658000 rw-p 00000000 00:00 0 01ea3000-022fe000 rw-p 00000000 00:00 0 [heap] 7f31a0c26000-7f31a0c3c000 r-xp 00000000 fd:00 524307 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f31a0c3c000-7f31a0e3b000 ---p 00016000 fd:00 524307 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f31a0e3b000-7f31a0e3c000 rw-p 00015000 fd:00 524307 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f31a0e3c000-7f31a0f68000 rw-p 00000000 00:00 0 7f31a0f68000-7f31a0f80000 r-xp 00000000 fd:00 535294 /lib/x86_64-linux-gnu/libpthread-2.19.so 7f31a0f80000-7f31a117f000 ---p 00018000 fd:00 535294 /lib/x86_64-linux-gnu/libpthread-2.19.so 7f31a117f000-7f31a1180000 r--p 00017000 fd:00 535294 /lib/x86_64-linux-gnu/libpthread-2.19.so 7f31a1180000-7f31a1181000 rw-p 00018000 fd:00 535294 /lib/x86_64-linux-gnu/libpthread-2.19.so 7f31a1181000-7f31a1185000 rw-p 00000000 00:00 0 7f31a1185000-7f31a1188000 r-xp 00000000 fd:00 535286 /lib/x86_64-linux-gnu/libdl-2.19.so 7f31a1188000-7f31a1387000 ---p 00003000 fd:00 535286 /lib/x86_64-linux-gnu/libdl-2.19.so 7f31a1387000-7f31a1388000 r--p 00002000 fd:00 535286 /lib/x86_64-linux-gnu/libdl-2.19.so 7f31a1388000-7f31a1389000 rw-p 00003000 fd:00 535286 /lib/x86_64-linux-gnu/libdl-2.19.so 7f31a1389000-7f31a1528000 r-xp 00000000 fd:00 535301 /lib/x86_64-linux-gnu/libc-2.19.so 7f31a1528000-7f31a1728000 ---p 0019f000 fd:00 535301 /lib/x86_64-linux-gnu/libc-2.19.so 7f31a1728000-7f31a172c000 r--p 0019f000 fd:00 535301 /lib/x86_64-linux-gnu/libc-2.19.so 7f31a172c000-7f31a172e000 rw-p 001a3000 fd:00 535301 /lib/x86_64-linux-gnu/libc-2.19.so 7f31a172e000-7f31a1732000 rw-p 00000000 00:00 0 7f31a1732000-7f31a179e000 r-xp 00000000 fd:00 524394 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f31a179e000-7f31a199e000 ---p 0006c000 fd:00 524394 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f31a199e000-7f31a199f000 r--p 0006c000 fd:00 524394 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f31a199f000-7f31a19a0000 rw-p 0006d000 fd:00 524394 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f31a19a0000-7f31a1b6b000 r-xp 00000000 fd:00 131288 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7f31a1b6b000-7f31a1d6b000 ---p 001cb000 fd:00 131288 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7f31a1d6b000-7f31a1d88000 r--p 001cb000 fd:00 131288 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7f31a1d88000-7f31a1d98000 rw-p 001e8000 fd:00 131288 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7f31a1d98000-7f31a1d9b000 rw-p 00000000 00:00 0 7f31a1d9b000-7f31a1df1000 r-xp 00000000 fd:00 133362 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7f31a1df1000-7f31a1ff1000 ---p 00056000 fd:00 133362 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7f31a1ff1000-7f31a1ff4000 r--p 00056000 fd:00 133362 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7f31a1ff4000-7f31a1ffb000 rw-p 00059000 fd:00 133362 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7f31a1ffb000-7f31a201b000 r-xp 00000000 fd:00 535260 /lib/x86_64-linux-gnu/ld-2.19.so 7f31a220b000-7f31a2210000 rw-p 00000000 00:00 0 7f31a2217000-7f31a221b000 rw-p 00000000 00:00 0 7f31a221b000-7f31a221c000 r--p 00020000 fd:00 535260 /lib/x86_64-linux-gnu/ld-2.19.so 7f31a221c000-7f31a221d000 rw-p 00021000 fd:00 535260 /lib/x86_64-linux-gnu/ld-2.19.so 7f31a221d000-7f31a221e000 rw-p 00000000 00:00 0 7fff1cf05000-7fff1cf2d000 rw-p 00000000 00:00 0 [stack] 7fff1cffc000-7fff1cffe000 r-xp 00000000 00:00 0 [vdso] 7fff1cffe000-7fff1d000000 r--p 00000000 00:00 0 [vvar] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] After ca. one hour it went yellow again with the following error output: "symlink nongreen.xml->index.wml failed: Transport endpoint is not connected". Shortly thereafter it went green again, but the runtime is about 1.5 times as high as before. xymon is installed from Debian's xymon packages. Feel free to tell me what else could be helpful to track down this bufferflow. I've found no (recent) core dump. Kind regards, Axel Beckert -- Axel Beckert <beckert at phys.ethz.ch> support: +41 44 633 26 68 IT Services Group, HPT H 6 voice: +41 44 633 41 89 Departement of Physics, ETH Zurich CH-8093 Zurich, Switzerland http://nic.phys.ethz.ch/

2 3

DATA messages
by dboldt＠usgs.gov 29 Sep '15

29 Sep '15

Hello, Pardon the newbie questions. I can't seem to find any references on the format for "data messages". Have tried sending the following, and no rrd file appears on the server: data igskahcmasmx01,cr,usgs,gov.msgRate messageRate: 1.3980 A second question is on best practices; How should one launch the data collecting program, with respect to the definition of the associated test? Below are the current pair of definitions I am using in clientlaunch.cfg: [msgRate] ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg CMD $XYMONCLIENTHOME/ext/msgRate LOGFILE $XYMONCLIENTLOGS/msgRate.log INTERVAL 5m [msgRate.data] ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg CMD $XYMONCLIENTHOME/ext/data/msgRate LOGFILE $XYMONCLIENTLOGS/msgRate-data.log INTERVAL 5m I'm assuming that I can't run multiple "CMD"s in a single task definition. Similarly I'm assuming that I can't reuse the same task identifier. -- -- David Boldt <dboldt at usgs.gov> "I stopped believing in Santa Claus when my mother took me to see him in a department store, and he asked for my autograph." --Shirley Temple (1928 - )

2 2

Display IP address on webpage
by jlaidman＠rebel-it.com.au 29 Sep '15

29 Sep '15

> > Is there a way to display the IP address instead of the hostname in the > overview webpages. I suppose I can use the NAME tag but it would mean I > have to add NAME tags to every host in host.cfg > Firsty, I'm curious why you would want to show the IP address and not the hostname. If the IP address is more useful, why aren't the devices named "device-192-168-0-100"? Secondly, do you know that you can click on an "info" link and get the IP addres? Thirdly, I'm not aware of a simple way to do what you're asking, certainly nothing built in to Xymon. Here are a few non-trivial options: 1. Modify the source code ;-) Actually, there are several features that I'd love to see configurable in xymongen, such as: a) hover or click to show a pop-up with some basic information about the server, perhaps some or all of the "info" page, or the contents of the "notes" page; b) configurable click-to-connect to launch telnet:// or ssh:// URLs; c) click to popup an offline/online option for the host; d) similar for shift-clicking on a status dot. Most, if not all of these, would have to be implemented in JavaScript, at least in part. But an option to expand some variables in the NAME: and DESCR: tags to include $IP would be coded directly in xymongen C code. 2. You could automate the generation of the NAME tags in hosts.cfg. Perhaps you could have a "hosts.src" file, and then you periodically create "hosts.cfg" such that it contains everything in "hosts.src" with an extra "NAME:192.168.0.100" type tag. 3. You could do a similar thing, but build a special include file containing only the IP address, hostname and alternative hostname, then use "dispinclude" to include this file BEFORE the real host description. Like this: *hosts.cfg:* dispinclude named-by-ip.cfg 192.168.0.100 host1.example.com # noconn ssh 192.168.1.53 host2.example.com # dns *named-by-ip.cfg:* 192.168.0.100 host1.example.com # NAME:192.168.0.100 192.168.1.53 host2.example.com # NAME:192.168.1.53 You would simply generate the file "named-by-ip.cfg" automatically whenever hosts.cfg is updated, or periodically. 4. You could probably do this sort of thing with some javascript magic. I'm guessing it would be possible to include a little javascript within the page footer, that scans through the <span title=> tags, performs a web lookup to the "info" column to get the IP address, and replaces the hostname text. I've done this kind of thing to allow hostnames to be clickable, enhancing the hostname text to be a hotlink to an "ssh://" or "telnet://" style URL. Described here: http://lists.xymon.com/archive/2012-March/034227.html

3 3

Color Status GIF for host status page
by steuer＠hotmail.com 29 Sep '15

29 Sep '15

Thanks Japheth! >From your hint I was able to modify my script to do what I wanted. So let me share now, maybe this will help others. As an example, using the dnsreg.sh script, I’ve modified 2 things: 1. With the widely published dnsreg.sh script, the EXPIRESTRING was showing like this for me: 2015-09-25T14 because it used a : as a delimiter, I removed the sed command and added an additional cut command using T as a delimiter so it only grabbed the actual date part. 2. Thanks to your hint, I added the DOTCOLOR commands in each part of the IF statement. Also in the last line I prefix with $DOTCOLOR so now it shows the colored icon in the detailed status page. EXPIRESTRING="`whois $DOMAIN | egrep -i "Expires.*:|Expiration.*:|Renewal.*:|Expires on" | head -n 1 | cut -d: -f2 | cut -dT -f1`" EXPIRES=`date --date="$EXPIRESTRING" "+%s"` if test $EXPIRES -le $ALARMTIME then COLOR=red DOTCOLOR='&red' elif test $EXPIRES -le $WARNTIME then COLOR=yellow DOTCOLOR='&yellow' else COLOR=green DOTCOLOR='&green' fi DOM=`echo $DOMAIN | sed -e 's/\./,/g'` $BB $BBDISP "status+12h $DOM.dnsreg $COLOR `date` $DOTCOLOR Domain $DOMAIN expires on: $EXPIRESTRING From: Japheth Cleaver [mailto:cleaver at terabithia.org] Sent: Friday, September 25, 2015 6:22 PM To: steuer at hotmail.com; xymon at xymon.com Subject: Re: [Xymon] Color Status GIF for host status page On 9/25/2015 2:36 PM, Rob Steuer wrote: Hi, I’ve tried to figure this out, but I need help at this point. What I’m trying to figure out is how to add a colored GIF status image to a detailed host page. For example, when looking at built in pages like Memory, Disk, Etc. The individual items show a small colored GIF image, green,yellow,red prior to the item [green gif] Physical: 80%. I’ve setup the UPS monitoring using devmon and the dnsreg script and while if I click on those pages it shows a green column/bar all the way to the left, it doesn’t show a green gif dot image next to the item that is tested. How can this be added? I’ve looked at the analysis.cfg and thought I was getting close, but I think it must be set/configured in some other way, it is defined on the client or the Xymon server and where? Rob, I can't speak to devmon *specifically* but generically speaking, the contents of the status message are controlled by the sender. The hard way to put it in would be to add the HTML IMG tag by hand, but fortunately there's an easier method :) Adding a '&red', '&yellow' or other &color to your status message on a line will cause the web interface to insert the appropriate color GIF for you automatically. HTH, -jc

2 1

graphing question
by bb4＠richter-it.net 28 Sep '15

28 Sep '15

Hi fellow XYmonians, I have a script that periodically checks my DSL speed and reports 2 values: Download : 13400801 Upload : 1027604 which are Bit/s. Actually it reports MBit/s and I recalculate the Bits. Now I want to graph these values. This is what I have in graphs.cfg: [speed] TITLE DSL Speed Info YAXIS Mbit/s DEF:Download=speed.rrd:Download:AVERAGE DEF:Upload=speed.rrd:Upload:AVERAGE LINE2:Download#00CCCC:Download LINE2:Upload#FF0000:Upload COMMENT:\n GPRINT:Download:LAST:Download \: %2.2lf%s (cur) GPRINT:Download:MAX:Download \: %2.2lf%s (max) GPRINT:Download:MIN:Download \: %2.2lf%s (min) GPRINT:Download:AVERAGE:Download \: %2.2lf%s (avg)\n GPRINT:Upload:LAST:Upload \: %2.2lf%s (cur) GPRINT:Upload:MAX:Upload \: %2.2lf%s (max) GPRINT:Upload:MIN:Upload \: %2.2lf%s (min) GPRINT:Upload:AVERAGE:Upload \: %2.2lf%s (avg)\n and this is in xymonserver.cfg: TEST2RRD="...,speed=ncv,..." GRAPHS="...,speed" NCV_speed="Download:GAUGE,Upload:GAUGE" The test runs every 20 minutes and I can see in speed.rrd the values on top of the file: <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">  <rrd> <version>0003</version> <step>300</step>  <lastupdate>1443085347</lastupdate>  <ds> <name> Download </name> <type> GAUGE </type> <minimal_heartbeat>600</minimal_heartbeat> <min>NaN</min> <max>NaN</max>  <last_ds>13400801</last_ds> <value>NaN</value> <unknown_sec> 147 </unknown_sec> </ds> <ds> <name> Upload </name> <type> GAUGE </type> <minimal_heartbeat>600</minimal_heartbeat> <min>NaN</min> <max>NaN</max>  <last_ds>1027604</last_ds> <value>NaN</value> <unknown_sec> 147 </unknown_sec> </ds> But these values are not passed down to the "archive" section of the file. So I might be missing something or maybe my graph definition is wrong. Can someone with a little bit more knowledge of RRD shed some light on this. Thanks Torsten

3 3

Xymon alert to pagerduty
by zforray＠vcu.edu 28 Sep '15

28 Sep '15

A couple of departments at the university were testing alternatives to the old pager process and they tested pagerduty but instead decided to go with victorops. I set alerts to both and everything seemed to work as it should. Victorops uses much more complicated email addresses but seem to work just fine...for example: MAIL 529597a1-47ad-4308-80f0-(12-digit code removed from this post)+ webservices at alert.victorops.com On Sat, Sep 26, 2015 at 12:23 AM, Eli via Xymon <xymon at xymon.com> wrote: > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon > > > ---------- Forwarded message ---------- > From: Eli <eliap09 at yahoo.com> > To: xymon at xymon.com > Cc: > Date: Fri, 25 Sep 2015 22:15:31 -0600 > Subject: Xymon alert to pagerduty > Hello, > > Is there any one tried forward/integrate xymon alert to pagerduty? > > Thanks > > *Sent from my Verizon Wireless 4G LTE DROID* > > -- *Zoltan Forray* TSM Software & Hardware Administrator Xymon Monitor Administrator Virginia Commonwealth University UCC/Office of Technology Services www.ucc.vcu.edu zforray at vcu.edu - 804-828-4807 Don't be a phishing victim - VCU and other reputable organizations will never use email to request that you reply with your password, social security number or confidential personal information. For more details visit http://infosecurity.vcu.edu/phishing.html

1 0

Xymon alert to pagerduty
by eliap09＠yahoo.com 28 Sep '15

28 Sep '15

Hello, Is there any one tried forward/integrate xymon alert to pagerduty? Thanks Sent from my Verizon Wireless 4G LTE DROID

4 3

Xymon alert to pagerduty
by solitaryr＠gmail.com 26 Sep '15

26 Sep '15

Just look at the xymon help for alerts. You can either use a script or email to manage them when they trigger. You can use whatever language you are most comfortable using. You can also look at the alerts tab on Xymonton to see some examples: https://wiki.xymonton.org/doku.php/alerts On Sat, Sep 26, 2015 at 11:00 AM, Eli via Xymon <xymon at xymon.com> wrote: > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon > > > ---------- Forwarded message ---------- > From: Eli <eliap09 at yahoo.com> > To: Galen Johnson <Galen.Johnson at sas.com> > Cc: xymon at xymon.com > Date: Sat, 26 Sep 2015 08:59:36 -0600 > Subject: Re: Xymon alert to pagerduty > Thanks much Johnson. Yes PD has web hook. Do mind if you give me some > insight what you did on xymon side, what the script do and where did you > put the script. > > *Sent from my Verizon Wireless 4G LTE DROID* > > > Galen Johnson <Galen.Johnson at sas.com> wrote: > > I've not tried pager duty but I have done Slack. If PagerDuty has a > webhook or other web interface, it should be easy enough to create a script > that calls it. You may want to start here: > > > https://developer.pagerduty.com/documentation/howto/integration-tutorial > > > =G= > > > ------------------------------ > *From:* Eli <eliap09 at yahoo.com> > *Sent:* Saturday, September 26, 2015 12:15 AM > *To:* xymon at xymon.com > *Subject:* Xymon alert to pagerduty > > Hello, > > Is there any one tried forward/integrate xymon alert to pagerduty? > > Thanks > > *Sent from my Verizon Wireless 4G LTE DROID* > >

1 0