August 2011 - Xymon - Xymon mailman web

timediff
by malcolm.hunter＠gmx.com 02 Aug '11

02 Aug '11

> ----- Original Message ----- > From: Henrik Størner > Sent: 08/02/11 09:22 AM > To: xymon at xymon.com > Subject: Re: [Xymon] timediff > > Both the Xymon server and client machine (Windows) are syncing off the > > same router/source. The times look identical so where does the 138 > > seconds skew come from? > > First, see the explanation of the CLOCK setting in the analysis.cfg > man-page. > > The client inserts a timestamp in the client message, usually just > before it transmits the message to the Xymon server (not sure if that is > the case with BBWin, though - it might add the timestamp when it begins > collecting data, which could explain it if the eventlog scanning takes a > long time). This timestamp is then compared with the time that Xymon > receives the message. In your case, they are 138 seconds different. BBWin builds the data file as it performs the tests, so it will always depend on how long it takes to do all of the tests. I'll raise a bug report to put the [clock] section at the end of the file. Regards, Malcolm

2 1

Xymon PROC check fails on un-aligned ps(1) output
by cs＠schug.net 02 Aug '11

02 Aug '11

If have got a question regarding Xymon 4.3.3 (running on CentOS 5.6/x86_64). In order to monitor the existence of certain processes like rsyslogd(8) I have following process rule defined in analysis.cfg: CLASS=linux PROC "%^/sbin/rsyslogd -m 0$" This works fine as long as the columns in the output of ps(1) (more specific “ps -Aww -o pid,ppid,user,start,state,pri,pcpu,time,pmem,rsz,vsz,cmd” as defined in xymonclient-linux.sh) are all nicely aligned. PID PPID USER STARTED S PRI %CPU TIME %MEM RSZ VSZ CMD [...] 4620 4607 68 Jun 17 S 22 0.0 00:00:00 0.0 860 12348 hald-addon-keyboard: listening on /dev/input/event0 4709 1 root Jun 17 S 17 0.0 00:00:00 0.0 496 8540 /usr/bin/hidd --server 4739 1 root Jun 17 S 21 0.0 00:11:14 0.0 3576 300132 /sbin/rsyslogd -m 0 6894 1 root Jun 17 S 18 0.0 00:00:00 0.0 1540 122008 automount 6918 1 root Jun 17 S 24 0.0 00:00:08 0.0 1224 63544 /usr/sbin/sshd The trouble starts when the process in question runs long enough (as seen on a different machine) so it does fit the reserved columns for that specific field, disturbing to whole output (process runtime is just one example, I suppose any value growing big enough to not fit anymore the reserved space would do to exploit this behavior): PID PPID USER STARTED S PRI %CPU TIME %MEM RSZ VSZ CMD [...] 5377 1 root May 24 S 21 0.0 00:00:00 0.0 444 3816 /sbin/mingetty tty4 5378 1 root May 24 S 20 0.0 00:00:00 0.0 444 3816 /sbin/mingetty tty5 5380 1 root May 24 S 19 0.0 00:00:00 0.0 444 3816 /sbin/mingetty tty6 5382 1 root May 24 S 22 0.0 00:00:00 0.0 496 3824 /sbin/agetty 9600 ttyS1 vt100 8734 1 root Jun 20 S 21 7.7 3-06:51:29 0.1 48640 292664 /sbin/rsyslogd -m 0 20468 262 root Jul 19 S 24 0.0 00:04:01 0.0 0 0 [pdflush] In this case the above regex does not seem to match anymore, because (apparently) the matching starts at some fixed column value. Just for fun and to double check I enhanced the process rule set by another rule: CLASS=linux PROC "%^/sbin/rsyslogd -m 0$" PROC "%^[0-9]+ /sbin/rsyslogd -m 0$" After doing so, indeed the first rule still fails while the second rule matches. So apparently the last digit of the VSZ field of rsyslogd(8) sneaked into the CMD field and gets matched by the PROC check. Is this a known bug, and if yes is there a good workaround for that apart from invoking a wrapper script in xymonclient-linux.sh which mangels the output of ps(1) accordingly? Thanks in advance! -cs

1 2

&JSHOSTLIST in maint_header
by Carl.Melgaard＠STAB.RM.DK 02 Aug '11

02 Aug '11

Hi, I noticed this line in the 4.3.4 maint_header file, which gives a javascript error when I use the "bb-hosts-1.3cgi"-script from cnaude.mdns.org. Is it used for anything? /melgaard

2 3

timediff
by camis＠mweb.co.za 02 Aug '11

02 Aug '11

Hi all, Could someone explain why the following would be occuring ----- Tue Aug 02 08:30:43 2011 NOT ok yellow System clock is 138 seconds off (max 60) local: Tue Aug 02 08:30:43 2011 UTC: Tue Aug 02 06:30:43 2011 ----- Both the Xymon server and client machine (Windows) are syncing off the same router/source. The times look identical so where does the 138 seconds skew come from? Regards, Cami

2 1

FW: timediff
by redrocketed＠gmail.com 02 Aug '11

02 Aug '11

We had this problem on windows machines, bbwin was analyzing the windows event logs which took the length of time of the reported timediff. We had 100,000 log lines in the application log alone, these over-wrote themselves every 20-30 minutes, even when you set a rule for bbwin to ignore these logs it still must process the 100,000 lines in order to realize it can ignore them. Our solution was to stop the application (mssql) logging these useless "login succeeded" events to the application log. For a short term test you could unload msgs.dll on the client side (if possible) which I believe processes event logs, or you could clear or limit the event log. Steve -----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Cami Sent: Tuesday, August 02, 2011 8:53 AM To: xymon at xymon.com Subject: [Xymon] timediff Hi all, Could someone explain why the following would be occuring ----- Tue Aug 02 08:30:43 2011 NOT ok yellow System clock is 138 seconds off (max 60) local: Tue Aug 02 08:30:43 2011 UTC: Tue Aug 02 06:30:43 2011 ----- Both the Xymon server and client machine (Windows) are syncing off the same router/source. The times look identical so where does the 138 seconds skew come from? Regards, Cami ______________________________ > > _________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

1 0

Upgrading server to 4.3.4 - ok with clients still 4.2.3?
by Carl.Melgaard＠STAB.RM.DK 02 Aug '11

02 Aug '11

Hi, Can I still roll with the old clients from 4.2-branch even tho Im upgrading my server to 4.3.4? /melgaard

2 1

setting for red/yellow hosts to appear on top of page or group
by naveen＠lastninja.net 02 Aug '11

02 Aug '11

Hi, Is there a setting to have hosts with red/yellow appear at the top of a page or group? Thanks, Naveen Nathan

3 2

Alerts do not seem to be triggering or otherwise executing my script
by chris.allen＠affinesystems.com 01 Aug '11

01 Aug '11

Seemingly the same problem as this guy: http://lists.xymon.com/archive/2008-June/019952.html I've crawled the hell out of the official Xymon documentation as well as the wikibook, I can't find anything that tells me what's going on. There is an alert that should have been triggering (It's covered by the time displayed in the log output below). The shutdown message is me running /etc/init.d/hobbit restart in an attempt to make it aware of the problem. (Xymon is currently red) Is the hobbitd_alert test/debug command supposed to actually execute the script/MAIL rule? Knowing this would help me a little, but not a whole lot as the alert doesn't seem to be otherwise triggering anyway. I am quite puzzled and unable to otherwise intelligently test or debug this, doubly disappointing is that I'm rather fond of this monitoring system and would rather not be forced to discard it because of a lack of documentation and abundance of inexplicable behavior. I have tested running my script standalone but with identical context to what would happen if hobbit executed it, and it worked. I also have debugging commands in the script to let me know if it gets executed at all. It never does during a --test nor naturally as it should be given the currently red status of one of my tests. Disregard that I am not utilizing $RCPT, I eliminated that to reduce the change it could be something unexpected with variable. It wasn't, the script just isn't getting executed. What am I missing? Log and command output follows this line: /etc/hobbit/scripts/alert.sh #!/bin/sh echo `set` > /tmp/envvars /data/software/aws/dist/current/sns/bin/sns-publish "arn:aws:sns:us-east-1:871321084716:ops" --subject "Server alert from Hobbit/Xymon" --message "$BBALPHAMSG" > /tmp/debug-alerts 2>&1 hobbit-alerts.cfg HOST=* COLOR=red SCRIPT /etc/hobbit/scripts/alert.sh "ops" REPEAT=1h tail -10 page.log 2011-07-21 14:15:01 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:17:01 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:17:10 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:19:10 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:20:02 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:20:02 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:20:13 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:22:13 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:22:45 Tried to down BOARDBUSY: Invalid argument 2011-07-21 14:22:45 Got a shutdown message 2011-07-21 14:22:45 Terminated by signal 15 2011-07-21 14:23:09 Peer not up, flushing message queue 2011-07-21 14:23:09 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:25:03 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:25:03 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:26:12 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:27:46 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:28:46 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:29:15 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:29:15 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg Running bbcmd hobbitd_alert --debug --test live proc (live is a real server/hostname and proc is a test criteria we have) returns: dev:/var/log/hobbit# bbcmd hobbitd_alert --debug --test live proc 2011-07-21 14:31:21 Using default environment file /usr/lib/hobbit/client/etc/hobbitserver.cfg 2011-07-21 14:31:21 Opening file /usr/lib/hobbit/server/etc/bb-hosts 2011-07-21 14:31:21 Opening file /usr/lib/hobbit/server/etc/hobbit-alerts.cfg 2011-07-21 14:31:21 Opening file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:31:21 Cannot open configuration file /usr/lib/hobbit/server/etc/hobbit-holidays.cfg 2011-07-21 14:31:21 send_alert live:proc state 0 00007255 2011-07-21 14:31:21 send_alert live:proc state Paging 00007255 2011-07-21 14:31:21 Matching host:service:page 'live:proc:' against rule line 122 00007255 2011-07-21 14:31:21 *** Match with 'HOST=* COLOR=red' *** 2011-07-21 14:31:21 Found a first matching rule 00007255 2011-07-21 14:31:21 Matching host:service:page 'live:proc:' against rule line 122 00007255 2011-07-21 14:31:21 *** Match with 'HOST=* COLOR=red' *** 2011-07-21 14:31:21 repeat live|proc|script|"ops" at 0 2011-07-21 14:31:21 Alert for live:proc to "ops" 00007255 2011-07-21 14:31:21 Script alert with command '/etc/hobbit/scripts/alert.sh' and recipient "ops" 2011-07-21 14:31:21 No more secondary matching rule dev:/var/log/hobbit# tail ./notifications.log Thu Jul 21 12:22:01 2011 nfs-1.procs (75.101.157.144) chris.allen at affinesystems.com[123] 1311276121 300 Thu Jul 21 12:52:31 2011 live.conn (75.101.158.206) engineeering at affinesystems.com[122] 1311277951 500 Thu Jul 21 12:52:31 2011 live.conn (75.101.158.206) chris.allen at affinesystems.com[123] 1311277951 500 Thu Jul 21 12:52:31 2011 nfs-1.procs (75.101.157.144) engineeering at affinesystems.com[122] 1311277951 300 Thu Jul 21 12:52:31 2011 nfs-1.procs (75.101.157.144) chris.allen at affinesystems.com[123] 1311277951 300 Thu Jul 21 13:23:01 2011 live.conn (75.101.158.206) ops at affinesystems.com 1311279781 500 Thu Jul 21 13:23:01 2011 nfs-1.procs (75.101.157.144) ops at affinesystems.com 1311279781 300 Thu Jul 21 13:50:01 2011 ads2.procs (68.168.100.16) arn:aws:sns:us-east-1:871321084716:ops 1311281397 300 Thu Jul 21 13:52:09 2011 ads2.procs (68.168.100.16) "arn:aws:sns:us-east-1:871321084716:ops" 1311281521 300 Thu Jul 21 13:59:58 2011 ads4.procs (68.168.100.18) "ops" 1311281995 300 dev:/var/log/hobbit# tail ./rrd-data.log 2011-07-21 13:42:27 Cache flush completed 2011-07-21 13:42:37 Peer not up, flushing message queue 2011-07-21 14:04:33 Tried to down BOARDBUSY: Invalid argument 2011-07-21 14:04:33 Shutting down, flushing cached updates to disk 2011-07-21 14:04:33 Cache flush completed 2011-07-21 14:04:42 Peer not up, flushing message queue 2011-07-21 14:22:45 Tried to down BOARDBUSY: Invalid argument 2011-07-21 14:22:45 Shutting down, flushing cached updates to disk 2011-07-21 14:22:45 Cache flush completed 2011-07-21 14:22:53 Peer not up, flushing message queue dev:/var/log/hobbit# tail ./history.log 2011-07-21 14:04:42 Peer not up, flushing message queue 2011-07-21 14:22:51 Peer not up, flushing message queue 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads2.msgs - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads2.ssh - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads3.msgs - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/ads4.msgs - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/dev.files - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/dev.ports - color unchanged (purple) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/live.msgs - color unchanged (clear) 2011-07-21 14:32:47 Will not update /var/lib/hobbit/hist/vcr1.msgs - color unchanged (purple)

2 1

Missing mapping in docs/Renaming-430.txt upgrade document
by cs＠schug.net 01 Aug '11

01 Aug '11

You might want to add this to the upgrade document --- xymon-4.3.3/docs/Renaming-430.txt.orig 2011-03-08 18:20:28.000000000 +0100 +++ xymon-4.3.3/docs/Renaming-430.txt 2011-08-01 20:58:20.000000000 +0200 @@ -144,6 +144,7 @@ BBHOSTHISTLOG XYMONHOSTHISTLOG BBHOSTS HOSTSCFG BBHTML XYMONHTMLSTATUSDIR +BBLOCATION XYMONNETWORK BBLOGSTATUS XYMONLOGSTATUS BBLOGS XYMONRAWSTATUSDIR BBMAXMSGSPERCOMBO MAXMSGSPERCOMBO Cheers -cs

2 1

AIX memory checks
by mailinglist0＠skurfer.com 01 Aug '11

01 Aug '11

We’re seeing garbage data on some of our AIX (7.1) clients like this: red Thu Jun 23 15:29:28 EDT 2011 - Memory CRITICAL Memory Used Total Percentage &red Physical 18446744073709551177M 4096M18446744073709551606% &green Swap 40M 4096M 0% The ones that exhibit problems have Active Memory Expansion enabled. Is there anything we can do to get valid data? Should I report it as a bug? -- Rob McBroom <http://www.skurfer.com/>

4 5