March 2013 - Xymon - Xymon mailman web

Xymon alert DURATION fail
by chris.pretorius＠signio.co.za 08 Mar '13

08 Mar '13

Good day I have a test that determines I a remote web service is responding. The webservice is managed by someone outside of my environment. The test fails a couple of times during the day, manual tests confirm that the xymon test is ok. The webservice will fail for maybe 30 to 60 minutes then will restore. If the Xymon test runs while the service is unavailable, I will receive a alert and by the time I can investigate all is ok. I added the DURATION>30 or DURATION>1h to the alerts.cnf to only send me alerts if the service has been down for a period of time, but I still receive the alert as soon as Xymon detects the problem. Is there another way to delay alerts or do I have some error in my configs? My alerts.cfg ============== PAGE=%prod SERVICE=institution,wsdl MAIL cellno-xymon at sms.domain.co.za DURATION>1h REPEAT 30m COLOR=red FORMAT=SMS SCRIPT /usr/lib/xymon/server/ext/smsplus.sh cell-xymon at sms.domain.co.za DURATION>1h REPEAT 30m COLOR=red Set debug option in tasks.cfg CMD xymond_channel --channel=page --log=$XYMONSERVERLOGS/alert.log xymond_alert --checkpoint-file=$XYMONTMP/alert.chk --checkpoint-interval=600 –debug Alert.log ======= 10687 2013-03-07 09:29:30 xymond_alert: Got message 237 @@page#237/ domain.co.za|1362641370.822859|196.14.77.34|domain.co.za |wsdl|196.14.77.34|1362643170|red|green|1362641370|prod|555815|linux|linux|| 10687 2013-03-07 09:29:30 startpos 2171, fillpos 2171, endpos -1 10687 2013-03-07 09:29:30 Got page message from domain.co.za:wsdl 10687 2013-03-07 09:29:30 Alert status changed from 0 to 1 10687 2013-03-07 09:29:30 Found a first matching rule 10687 2013-03-07 09:29:30 No more secondary matching rule 10687 2013-03-07 09:29:30 No files modified, skipping reload of /usr/lib/xymon/server/etc/alerts.cfg 10687 2013-03-07 09:29:30 No files modified, skipping reload of /usr/lib/xymon/server/etc/holidays.cfg 10687 2013-03-07 09:29:30 Found a first matching rule 10687 2013-03-07 09:29:30 No more secondary matching rule 10687 2013-03-07 09:29:30 Found a first matching rule 10687 2013-03-07 09:29:30 1 alerts to go 12977 2013-03-07 09:29:30 send_alert domain.co.za:wsdl state 0 12977 2013-03-07 09:29:30 Found a first matching rule 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|mail| operations at domain.co.za at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to operations at domain.co.za 10687 2013-03-07 09:29:30 Found a first matching rule 10687 2013-03-07 09:29:30 No more secondary matching rule 10687 2013-03-07 09:29:30 Want msg 238, startpos 2171, fillpos 2171, endpos -1, usedbytes=0, bufleft=264068 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|mail|REPEAT at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to REPEAT 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|mail|30m at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to 30m 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|mail| cellno-xymon at sms.conclude.co.za at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to cellno-xymon at sms.conclude.co.za 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|mail|REPEAT at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to REPEAT 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|mail|30m at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to 30m 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|script| cellno-xymon at sms.conclude.co.za at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to cellno-xymon at sms.conclude.co.za 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|script|REPEAT at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to REPEAT 12977 2013-03-07 09:29:30 repeat domain.co.za|wsdl|script|30m at 0 12977 2013-03-07 09:29:30 Alert for domain.co.za:wsdl to 30m 12977 2013-03-07 09:29:30 No more secondary matching rule Kind Regards, Chris Pretorius Systems Administrator [image: logo small] *Your market gateway* * * Southdowns Office Park, Block B 2nd Floor SF02, Cnr John Vorster Drive & Karee Street, Highveld, Centurion Tel: 012 684 6600 [image: dot red] Fax: 012 684 6629 [image: dot red] Email: chris.pretorius at signio.co.za [image: dot red] Web: www.signio.co.za [image: 3] This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify us via email : support at signio.co.za . Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Finally, the recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. Signio (PTY) LTD, Reg Nr: 2003/001508/07

3 3

Change URL for Xymon mail alert report from http:// to https://
by jje＠fynbus.dk 08 Mar '13

08 Mar '13

Hello I tried search the internet and looked on wiki xymon for how to change the mail alert url from http// to https:// format in xymon But I haven't found anything for in which file this is controlled and can be change Any help appreciated Med venlig hilsen / best regards Jesper Jensen

3 2

Problem with netapp.pl
by linuxmasterjedi＠free.fr 08 Mar '13

08 Mar '13

Hi, I've just contacted "Francesco Duranti" who maintain some plugins for BEA/Netapp and DB [1] At the same time, I would to know if someone could be interested by implementing a new feature in this perl code ? On the netapp.ini file, we can add exception for Disk (D) warning/panic level, Inode (I) warning/panic level, Snapshot (S) warning/panic level but we can add exceptions to Quota. I'm looking for someone who could duplicate/tweak a couple of perl code lines to add this Quota exception feature. Contact me for extra information, thanks ! [1] : http://sourceforge.net/projects/hobbit-perl-cl/

1 1

problem with ignoring logs from bbwin clients in central mode
by Phil.Crooker＠orix.com.au 08 Mar '13

08 Mar '13

I'm just setting up a number of windows servers with bbwin clients and am having the problem of not being able to stop sending log files or ignorned entries to the xymon server. The logs are often overwhelmingly filled with junk which makes it hard to see the interesting bits. In analysis.cfg, it says: # To monitor a logfile, you *MUST* configure both client-local.cfg # and analysis.cfg. If you configure only the client-local.cfg # file, the client will collect the log data and you can view it in # the "client data" display, but it will not affect the color of the # "msgs" status. On the other hand, if you configure only the # analysis.cfg file, then there will be no log data to inspect, # and you will not see any updates of the "msgs" status either. This is the case with the xymon linux client but not with bbwin clients set to central mode (not sure about local mode but I haven't fully investigated). Even if I have nothing in client-local.cfg [win32] clientclass (that of the client), all log files are sent to the xymon server anyway. Here is the windows clientclass: [win32] log:eventlog_application IGNORE BigBrotherXymonClient Here is what I have for a linux client: [linux] log:/var/log/messages:10240 ignore SQLAnywhere Using tcpdump, when the linux client starts, it does a syn/ack with the server and immediately sends its data, including the 'ignored' lines. At the end of that, the server sends the above clientclass statement. Next time the client data is sent to the xymon server, the ignored lines aren't sent. But at no point are the ignored lines displayed on the msgs web page. With the bbwin client, the same sequence happens, but the xymon server doesn't send the win32 segment and all logs and all log entries are always sent. I confirmed in the tcpdump that the client is sending win32: whatever.domain.bbwin win32 [osversion] Microsoft Windows Server 2003, Stadard Edition Service Pack 2 (build.3790) ..... Am I doing something wrong here or is the central mode for this client not fully implemented, or ??? thanks, Phil

1 0

regular expressions in analysis.cfg
by Phil.Crooker＠orix.com.au 07 Mar '13

07 Mar '13

Hi David, Thanks, it does work on ignore statements with regular expressions but not with simple strings: LOG eventlog_application %^warning COLOR=yellow "IGNORE=No externals have been specified" LOG eventlog_application %^warning COLOR=yellow IGNORE="No externals have been specified" If I escape the spaces or if I add the %, it works. But still, this is not logical - quotes are normally after the equal sign and of course it shouldn't be necessary to make this into a regex when it isn't. thanks for the tip. cheers, Phil >>> David Baldwin <david.baldwin at ausport.gov.au> 03/04/13 4:43 PM >>> Phil, You need to put "" around entire IGNORE item - e.g.: ... "IGNORE=%Client \([0-9]7" This applies for any use of regex in analysis.cfg which uses spaces. David. I'm having a problem with spaces in regular expressions when trying to IGNORE log entries. So for the line: warning - 2013/03/04 12:52:21 - BigBrotherXymonClient (17) - The agent externals generated this event message : No externals have been specified works: LOG eventlog_application %^warning COLOR=red IGNORE=%Client.*17 works: LOG eventlog_application %^warning COLOR=red IGNORE=%Client.*[0-9]7 works: LOG eventlog_application %^warning COLOR=red IGNORE=Client\ \(17 works: LOG eventlog_application %^warning COLOR=red IGNORE=No\ externals\ have\ been\ specified doesn't work: LOG eventlog_application %^warning COLOR=red IGNORE=%"Client \([0-9]7" doesn't work: LOG eventlog_application %^warning COLOR=red IGNORE="Client \(17" doesn't work: LOG eventlog_application %^warning COLOR=red IGNORE="Client (17" doesn't work: LOG eventlog_application %^warning COLOR=red IGNORE="No externals have been specified" Is this correct? If so, why no spaces??? It is very confusing as spaces are used elsewhere, eg I can have spaces in a DISK name in analysis.cfg: DISK "/yada yada" 90 95, and spaces can be used in various places in hosts.cfg. thanks, Phil _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon -- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 david.baldwin at ausport.gov.au Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.

2 3

Debug for client side scripts
by Michael.Beatty＠sherwin.com 07 Mar '13

07 Mar '13

Is there a variable that my client side scripts can access to know if the xymon client was started in debug mode? I would like my client scripts to write to their log files dependent on if the client is running in debug or not. Only logging true errors normally, logging debug messages when in debug. -- Michael Beatty

2 1

New monitoring script on Xmonton - db_cpu.ksh
by a.chervonets＠cominder.eu 07 Mar '13

07 Mar '13

Good news. Had reviewed the script. There may be one problem in that lines: -- for PROC in $(ps -efa | grep $DB | awk '{ print $2 }') -- You do just grep, but imagine there may be instances like: MYDB MYDB1 MYDB12 or NOMYDB So it will count MYDB numbers for MYDB1, MYDB12 and NOMYDB too We should make grepping more intelligent, because oracle processes may be listed like: ora_j000_MYDB oracleMYDB (LOCAL=NO) ora_ckpt_MYDB and so on Best regards, Andrey Chervonets ---------------------- CoMinder Ltd. http://www.cominder.eu/ Mobile: +371 26517848 Fax: +371 66066346 On 01.03.2013 13:00, xymon-request at xymon.com wrote: > Message: 10 > Date: Fri, 1 Mar 2013 14:40:58 +0800 > From: Vernon Everett<everett.vernon at gmail.com> > To: Xymon mailinglist<xymon at xymon.com> > Subject: [Xymon] New monitoring script on Xmonton > Message-ID: > <CAGo4kcYUaYJ8TSjrs4xf7pNzptsB3vk3B5omLAae_M-2jtWokg at mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi all > > I have just put something together to monitor Oracle database CPU usage. > http://www.xymonton.org/monitors:db_cpu.ksh

4 4

sepparated disk alerts (OPEN)
by oyvind.bjorge＠telenor.com 06 Mar '13

06 Mar '13

Hi Sorry for late response to this. A suggestion regarding your problem with max disable time. (In this case I consider the changing of threshold to be a disabling.) For the use of a web-gui I would create a separate configfile where I keep track of these changes. This would have one line for all active disabled volumes with info like nodename, volume, timestamp for the disabling, and maybe a timestamp for normalizing. This configfile could be used by the webpage so it not only gives possibility to disable volumes, but always also show the actively disabled volumes (and with a button for normalizing). In addition you should have a small job that runs periodically and check if timestamp for normalizing has been reached, or if timestamp of disabling is older than a defined max time (ex 28 days). Also you could create a client-test on the server checking the config-file and generate alarms to xymon based on the time. Something in the line of this should be possible to implement without too much work without having to change anything on the clients, keeping the requirement of not forgetting them. Than you just have to get permission from management to do these kind of changes to the server..... - Oyvind -----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Aiquen Sent: 20. februar 2013 05:00 To: Asif Iqbal Cc: xymon at xymon.com Subject: Re: [Xymon] sepparated disk alerts Hi Thanks for the suggestion. Oyvind suggestion seems close to what I need. I only need to find a way to make it keep track of which nodes have their limits raised. I am not allowed to make changes to the levels of the alerts without making sure that the levels get droped down again after a certain time. This is laid down as a rule from higher ups in my company. What Asif and Ralph suggests is also good, but would breach the rule about "NO mails may be automatically sent from xymon anywhere" that is laid down on me. Thank you for good suggestions and sorry about saying that they don't work for me. I want to point out that these will probably technically work to solve the problem. But I am not allowed to implement them because of the set of rules I have to follow that I mentioned in my first post. That is why I called this more of a new feature request than an actual issue. Pritty much the only acceptable solution for the higher ups is that a new client is released with the feature to sepparate disk alerts based on monitored disks. And then be able do disable or raise the limit within a time boundury, since we have to be able to garantee that the alert will come back and remind us if no one lookes in to it for a certain amount of time. Maby it is worth to mention that the feature "disable until ok" is disabled from our xymon. And you cannot disable an alert for more than 28 days to make sure that no alert ever gets neglected. Again, thank you for your time. All suggestions is apprisiated. I will try to work on something along the lines of Oyvinds suggestion. Kind Regards Calle Lejdbrandt On Tue, Feb 19, 2013 at 5:02 AM, Asif Iqbal <vadud3 at gmail.com> wrote: > On Fri, Feb 15, 2013 at 3:14 PM, Aiquen <aiqueneldar at gmail.com> wrote: >> Greetings >> >> I don't know if this is the correct forum for this question or more >> like it, new feature request. How ever I'm gonna post here and hope >> someone will point me right if this is the wrong place. >> >> For the issue at hand. At my company we use Xymon to monitor >> thousands of servers. And sometimes disks gets filled and thus >> generates an alert. Sometimes this won't get looked into for a couple >> of days since our clients have specified that they want to remove >> data themselves (and not buy more storage). And as several servers >> have 3 - 8 disk and/or partitions we sometimes have trouble >> monitoring the other disks since one is already sending an alert. >> >> Example: >> Server1 has 3 monitored partitions of a disk: >> / >> /srv/important/data >> /srv/database >> >> with the same limits on all three: 80% -> yellow alert; 90% -> red >> alert Then if /srv/important/data reaches 82% the client wants us to >> notify them and they will free up space. This normaly takes around 3 >> - 5 working days. But they also wants us to monitor /srv/database. >> And say that 1 day after /srv/important/data gets filled, >> /srv/database reaches 84%. That will not trigger a new alert in the >> non-green status view which is what we monitor. >> >> The question/request is then as this: Is there a way to get the >> client to report each disk/partition as a separate alert, so we can >> disable the alert for one disk while receiving alerts for the other >> disks/partions. To use the example I want to be able to temporarily >> set /srv/important/data in disabled mode while still getting alerts >> from /srv/database > > analysis.cfg > > HOST=myhost > DISK /srv/database GROUP=A > DISK /srv/important/data GROUP=B > > alerts.cfg > > GROUP=A COLOR=red > MAIL groupA > > GROUP=B COLOR=red > MAIL groupB > > > > This might be start. > > >> >> I know this could be solved by writing my own script for the client, >> but that was disapproved of from management as they want as few >> custom scripts to maintain as possible (we already have dozens of >> custom scripts). >> >> All help and feedback is appreciated, thanks. >> >> Kind regards >> Calle Lejdbrandt >> _______________________________________________ >> Xymon mailing list >> Xymon at xymon.com >> http://lists.xymon.com/mailman/listinfo/xymon > > > > -- > Asif Iqbal > PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

2 1

Feature Request ~ logfetch
by Michael.Beatty＠sherwin.com 06 Mar '13

06 Mar '13

I see the comments made in logfetch.c /* Is it ok for these to be hardcoded ? */ #define MAXCHECK 102400 /* When starting, dont look at more than 100 KB of data */ #define MAXMINUTES 30 #define POSCOUNT ((MAXMINUTES / 5) + 1) #define LINES_AROUND_TRIGGER 5 My answer to this would be, "no". Due to various reasons, it is not desirable to run my client more than once every hour. Therefore, the msgs check for log files is almost useless unless I change these values (and honestly, I'm not even sure that would work as I haven't seen if this would impact anything else). If I ran my client every hour, I would lose 50% of my log data. I would like to see where the log check would be dependent on how long of an interval your client is run as opposed to a hard coded 30 minutes. -- Michael Beatty

4 3

[Newbie] Understanding client-server configuration (terabithia's rpm 4.3.10-1.fc17)
by a_s_y＠sama.ru 06 Mar '13

06 Mar '13

Hello. I attempt to setup xymon from terabithia's packages (rebuilded 4.3.10-1.fc17.src.rpm with small changes for ALT Linux) Xymon's server works and ready for receive data from clients I think: tcp 0 0 0.0.0.0:1984 0.0.0.0:* LISTEN 3557/xymond Web pages generated correctly. All server's tests works. Now I attempt to setup client. So, 3 questions. 1. I need to make some network's tests on client and transfer results to server. Should I setup client installation or server installation in this case ? 2. I found a bug in packages. xymon and xymon-client packages contains /usr/libexec/xymon-client/xymonclient.sh with code: Is this script need always ? if test "$LOCALMODE" = "yes"; then echo "@@" >> $MSGTMPFILE /usr/libexec/xymon-client/xymond_client --local <$MSGTMPFILE else xymond_client exist in xymon package only and by another path: /usr/libexec/xymon/xymond_client Is this script need always ? Could this be the cause of problems in the client ? 3. About package. xymon package contains xymonclient-*.sh for non linux. All this removed from xymon-client. Are they forgotten in xymon package or they needed ? -- Regards, Sergey

3 9