July 2011 - Xymon - Xymon mailman web

use of aberrant behavior detection
by Mario.Contreras＠tigo.com.co 06 Aug '11

06 Aug '11

Hi We are using xymon/devmon pair and we want to use for some tests the "aberrant behavior detection" of RRDTOOL. Could someone please tell me what changes are necessary in XYMON configuration files to do so? I saw a similar question in the mail list back in 2005 but with no answer. Best regards Mario Contreras mario.contreras at tigo.com.co <mailto:mario.contreras at tigo.com.co> Este mensaje (incluyendo cualquier anexo) contiene informaci�n confidencial que se encuentra protegida por la ley. La informaci�n que contiene s�lo puede ser utilizada por la persona o la compa��a a la cual est� dirigido. Si usted no es un receptor autorizado, o por error recibe el mensaje, omita su contenido y elim�nelo inmediatamente. Cualquier retenci�n, difusi�n, distribuci�n y/o copia as� como la ejecuci�n de cualquier acci�n basado en el contenido del mismo, se encuentra estrictamente prohibido, y esta amparado por la ley. -------------- This message (including any attachments) contains confidential information intented for a specific individual and purpose, and is protected by law. If you are not the intented recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.

2 3

Apache tag not working, not playing nice with http tag.
by betsy.schwartz＠gmail.com 04 Aug '11

04 Aug '11

I've been trying to get apache results and am getting nothing, no results, no errors, as if it the tag wasn't there. UNLESS, and this is the weird part - *IF* I disable http for a given server I get *stats* under apache on the trends page, but still no apache column on the server page. Disabling http is not an option, as we've got nonstandard ports getting tested. With any http test enabled, just get blank "nan" values for apache. And no matter what, we don't get an apache column. Xymon server is xymon 4.3.2 on Oracle Enterprise Linux; most of the clients are running the latest apache and Solaris 10 u9. servers have ExtendedStatus On curl on the xymon server returns the expected stats with "curl http://myserver.example.com/server-status?auto" have tested using just a plain "apache" tag in hosts.cfg and apache=http://myserver.example.com/server-status?auto

3 6

two tests showing purple and clear
by Paul.A.Gallegos＠wellsfargo.com 02 Aug '11

02 Aug '11

Ok, I'm stumped. I'm running Xymon 4.3.2 on a group of 10 RedHat 5.4 x64 servers, 4 of which are DB servers. 3 out of the four DB servers are showing green, the fourth shows clear for msgs (it says "no data reported") and purple for processes (over 20 days since something was reported). However, a ps command shows that the processes it should be checking for exist, and I can tail the log to see things scrolling through. I reinstalled the client on that server and it still won't go green. The configuration for all four DB servers is the same, outside of one script in ext/ directory. What could possibly be the problem?

2 1

Alerts do not seem to be triggering or otherwise executing my script
by chris.allen＠affinesystems.com 01 Aug '11

01 Aug '11

Seemingly the same problem as this guy: http://lists.xymon.com/archive/2008-June/019952.html I've crawled the hell out of the official Xymon documentation as well as the wikibook, I can't find anything that tells me what's going on. There is an alert that should have been triggering (It's covered by the time displayed in the log output below). The shutdown message is me running /etc/init.d/hobbit restart in an attempt to make it aware of the problem. (Xymon is currently red) Is the hobbitd_alert test/debug command supposed to actually execute the script/MAIL rule? Knowing this would help me a little, but not a whole lot as the alert doesn't seem to be otherwise triggering anyway. I am quite puzzled and unable to otherwise intelligently test or debug this, doubly disappointing is that I'm rather fond of this monitoring system and would rather not be forced to discard it because of a lack of documentation and abundance of inexplicable behavior. I have tested running my script standalone but with identical context to what would happen if hobbit executed it, and it worked. I also have debugging commands in the script to let me know if it gets executed at all. It never does during a --test nor naturally as it should be given the currently red status of one of my tests. Disregard that I am not utilizing $RCPT, I eliminated that to reduce the change it could be something unexpected with variable. It wasn't, the script just isn't getting executed. What am I missing? Log and command output follows this line: /etc/hobbit/scripts/alert.sh #!/bin/sh echo `set` > /tmp/envvars /data/software/aws/dist/current/sns/bin/sns-publish "arn:aws:sns:us-east-1:871321084716:ops" --subject "Server alert from Hobbit/Xymon" --message "$BBALPHAMSG" > /tmp/debug-alerts 2>&1 hobbit-alerts.cfg HOST=* COLOR=red SCRIPT /etc/hobbit/scripts/alert.sh "ops" REPEAT=1h tail -10 page.log 2011-07-21 14:15:01 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:17:01 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:17:10 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:19:10 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:20:02 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:20:02 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:20:13 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:22:13 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:22:45 Tried to down BOARDBUSY: Invalid argument 2011-07-21 14:22:45 Got a shutdown message 2011-07-21 14:22:45 Terminated by signal 15 2011-07-21 14:23:09 Peer not up, flushing message queue 2011-07-21 14:23:09 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:25:03 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:25:03 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:26:12 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:27:46 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:28:46 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:29:15 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:29:15 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg Running bbcmd hobbitd_alert --debug --test live proc (live is a real server/hostname and proc is a test criteria we have) returns: dev:/var/log/hobbit# bbcmd hobbitd_alert --debug --test live proc 2011-07-21 14:31:21 Using default environment file /usr/lib/hobbit/client/etc/hobbitserver.cfg 2011-07-21 14:31:21 Opening file /usr/lib/hobbit/server/etc/bb-hosts 2011-07-21 14:31:21 Opening file /usr/lib/hobbit/server/etc/hobbit-alerts.cfg 2011-07-21 14:31:21 Opening file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:31:21 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:31:21 send_alert live:proc state 0 00007255 2011-07-21 14:31:21 send_alert live:proc state Paging 00007255 2011-07-21 14:31:21 Matching host:service:page 'live:proc:' against rule line 122 00007255 2011-07-21 14:31:21 *** Match with 'HOST=* COLOR=red' *** 2011-07-21 14:31:21 Found a first matching rule 00007255 2011-07-21 14:31:21 Matching host:service:page 'live:proc:' against rule line 122 00007255 2011-07-21 14:31:21 *** Match with 'HOST=* COLOR=red' *** 2011-07-21 14:31:21 repeat live|proc|script|"ops" at 0 2011-07-21 14:31:21 Alert for live:proc to "ops" 00007255 2011-07-21 14:31:21 Script alert with command '/etc/hobbit/scripts/alert.sh' and recipient "ops" 2011-07-21 14:31:21 No more secondary matching rule dev:/var/log/hobbit# tail ./notifications.log Thu Jul 21 12:22:01 2011 nfs-1.procs (75.101.157.144) chris.allen at affinesystems.com[123] 1311276121 300 Thu Jul 21 12:52:31 2011 live.conn (75.101.158.206) engineeering at affinesystems.com[122] 1311277951 500 Thu Jul 21 12:52:31 2011 live.conn (75.101.158.206) chris.allen at affinesystems.com[123] 1311277951 500 Thu Jul 21 12:52:31 2011 nfs-1.procs (75.101.157.144) engineeering at affinesystems.com[122] 1311277951 300 Thu Jul 21 12:52:31 2011 nfs-1.procs (75.101.157.144) chris.allen at affinesystems.com[123] 1311277951 300 Thu Jul 21 13:23:01 2011 live.conn (75.101.158.206) ops at affinesystems.com 1311279781 500 Thu Jul 21 13:23:01 2011 nfs-1.procs (75.101.157.144) ops at affinesystems.com 1311279781 300 Thu Jul 21 13:50:01 2011 ads2.procs (68.168.100.16) arn:aws:sns:us-east-1:871321084716:ops 1311281397 300 Thu Jul 21 13:52:09 2011 ads2.procs (68.168.100.16) "arn:aws:sns:us-east-1:871321084716:ops" 1311281521 300 Thu Jul 21 13:59:58 2011 ads4.procs (68.168.100.18) "ops" 1311281995 300 dev:/var/log/hobbit# tail ./rrd-data.log 2011-07-21 13:42:27 Cache flush completed 2011-07-21 13:42:37 Peer not up, flushing message queue 2011-07-21 14:04:33 Tried to down BOARDBUSY: Invalid argument 2011-07-21 14:04:33 Shutting down, flushing cached updates to disk 2011-07-21 14:04:33 Cache flush completed 2011-07-21 14:04:42 Peer not up, flushing message queue 2011-07-21 14:22:45 Tried to down BOARDBUSY: Invalid argument 2011-07-21 14:22:45 Shutting down, flushing cached updates to disk 2011-07-21 14:22:45 Cache flush completed 2011-07-21 14:22:53 Peer not up, flushing message queue dev:/var/log/hobbit# tail ./history.log 2011-07-21 14:04:42 Peer not up, flushing message queue 2011-07-21 14:22:51 Peer not up, flushing message queue 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads2.msgs - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads2.ssh - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads3.msgs - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads4.msgs - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/dev.files - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/dev.ports - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/live.msgs - color unchanged (clear) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/vcr1.msgs - color unchanged (purple)

2 1

AIX memory checks
by mailinglist0＠skurfer.com 01 Aug '11

01 Aug '11

We’re seeing garbage data on some of our AIX (7.1) clients like this: red Thu Jun 23 15:29:28 EDT 2011 - Memory CRITICAL Memory Used Total Percentage &red Physical 18446744073709551177M 4096M18446744073709551606% &green Swap 40M 4096M 0% The ones that exhibit problems have Active Memory Expansion enabled. Is there anything we can do to get valid data? Should I report it as a bug? -- Rob McBroom <http://www.skurfer.com/>

4 5

Acknowledge Alert web page - patch for durations
by keenha＠googlemail.com 01 Aug '11

01 Aug '11

Hi all, For anyone that is interested ... I've written a patch for the Acknowledge Alert page so that you can select a duration in mins, hours or days (for those of us who have long term problems!). Thought I'd share it with y'all. Apply to v4.3.3 web/acknowledge.c: 15a16,19 > /*----------------------------------------------------------------------------------*/ > /* Edited by Heather Keen 2011/06/29, to add different period types to ACK duration.*/ > /*----------------------------------------------------------------------------------*/ > 36a41 > char *period; 44a50 > char *periodall = NULL; 76c82 < char *acknum = NULL, *validity = NULL, *ackmsg = NULL; --- > char *acknum = NULL, *validity = NULL, *ackmsg = NULL, *period = NULL; 86a93,95 > else if (strcasecmp(cwalk->name, "PERIOD_all") == 0) { > if (*cwalk->value) periodall = cwalk->value; > } 93a103,106 > else if (strncasecmp(cwalk->name, "PERIOD", 5) == 0) { > if (*cwalk->value) period = cwalk->value; > delim = strchr(cwalk->name, '_'); if (delim) id = atoi(delim+1); > } 139a153 > if (period) awalk->period = strdup(period); 164c178,179 < fprintf(output, " <TD><INPUT TYPE=TEXT NAME=\"DELAY_%s\" SIZE=8 MAXLENGTH=20></TD>\n", numstr); --- > fprintf(output, " <TD NOWRAP><INPUT TYPE=TEXT NAME=\"DELAY_%s\" SIZE=4 MAXLENGTH=6><SELECT NAME=\"PERIOD_%s\"><OPTION VALUE=min SELECTED>min(s)<OPTION VALUE=hour>hour(s)<OPTION VALUE=day>day(s)</SELECT></TD>\n", numstr, numstr); > 275c290 < fprintf(stdout, "<tr><th align=left>Host</th><th align=left>Test</th><th align=left>Duration<br>(minutes)</th><th align=left>Cause</th><th>Ack</th><th>Ack Multiple</tr>\n"); --- > fprintf(stdout, "<tr><th align=left>Host</th><th align=left>Test</th><th align=left>Duration</th><th align=left>Cause</th><th>Ack</th><th>Ack Multiple</tr>\n"); 326a342 > if (periodall) awalk->period = periodall; 328a345,347 > if (strncmp(awalk->period, "hour", 4) == 0) awalk->validity *= 60; > else if (strncmp(awalk->period, "day", 4) == 0) awalk->validity *= 60*24; > Regards, Heather

2 1

Add page name to alert
by arjunp＠qualcomm.com 01 Aug '11

01 Aug '11

Hi , A newbie question: I have hosts categorized in pages (in hosts.cfg) and want the email alerts to show the page name in the subject. Can this be done? Thanks, Arjun

2 1

disk reporting issue on xymon 4.3.0-beta2
by tlewick＠tradebotsystems.com 31 Jul '11

31 Jul '11

Has anyone seen similar issues as below? Very rarely, we get false positives due to disk and process data coming from hosts being somewhat garbled. However, across several hundred hosts, it happens just often enough to be annoying. Below is an example of the disk issue from one of our xymon clients. As you can see, the disks were actually not above 30% utilized, and the alert cleared the next cycle. Just wondering if this has been resolved in later xymon versions? We are running xymon 4.3.0-0.beta2 Wed Jul 6 08:0]:30 CDT 2011 - Filesystems NOT ok 22% / (102738628% used) has reached the PANIC level (97%) 29% /boot (351664% used) has reached the PANIC level (97%) Filesystem ] 1]24-blocks Used Available Capacity Mounted on /dev/sda7 137886784 28143896 102738628 22% / /dev/sda6 521064 142932 351664 29% /boot Next update cycle, everything is fine. Wed Jul 6 08:11:31 CDT 2011 - Filesystems OK Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda7 137886784 28144080 102738444 22% / /dev/sda6 521064 142932 351664 29% /boot

4 9

Payed support
by jef.jagers＠thomsonreuters.com 31 Jul '11

31 Jul '11

Hi Hendrik, Jeff, Although my google search probably provided me with an answer to my question I wanted to ask you just to make sure. Is there any payed support available for xymon? Regards, Jef Jagers

2 1

Patch: rrd disk fix
by malcolm.hunter＠gmx.com 30 Jul '11

30 Jul '11

Hi, This is a patch to fix a bug with parsing of disk data for RRD graphing. The original code assumed that if the drive letter was longer than one character, the system was actually a UNIX system. In fact, this column also holds the directory where volumes can be mounted - just like with UNIX systems. The patch removes this restriction - so it now only checks if there's a '/' at the start of the first column. Regards, Malcolm Index: xymond/rrd/do_disk.c =================================================================== --- xymond/rrd/do_disk.c (revision 6701) +++ xymond/rrd/do_disk.c (working copy) @@ -103,8 +103,7 @@ * So check if there's a slash in the NT filesystem letter - if yes, * then it's really a Unix system after all. */ - if ( (dsystype == DT_NT) && (*(columns[5])) && - ((strchr(columns[0], '/')) || (strlen(columns[0]) > 1)) ) + if ( (dsystype == DT_NT) && (*(columns[5])) && (strchr(columns[0], '/')) ) dsystype = DT_UNIX; switch (dsystype) {

2 1