January 2006 - Xymon - Xymon mailman web

Alert TIME problem
by whk2＠cornell.edu 27 Jan '06

27 Jan '06

Hi Everyone, I just switched from BB a couple of weeks ago. We are running version 4.1.2p1 on RHLinux AS v3. I am aware that the DURATION parameter does not work. However, I am also having a TIME parameter problem. I want to run two different alerts. One with the REPEAT setting and one without all based upon the TIME. Below is how I have the settings: HOST=xxxx.cornell.edu MAIL xxx at xxx.cornell.edu DURATION>=5 COLOR=red REPEAT=60 RECOVERED TIME=*:0700:2100 MAIL xxx at xxx.cornell.edu DURATION>=5 COLOR=red RECOVERED TIME=*:2101:0659 However for the alerts that occurred last night I received REPEAT messages all night long. I want to receive any red messages for any category I am monitoring. Does anyone see a problem or know of a TIME work around? If anyone knows of a work around for the DURATION problem I would be most appreciative. Thanks, Bill

1 0

[hobbit] Latest Snapshot
by fduranti＠q8.it 27 Jan '06

27 Jan '06

Hi Henrik, Thanks for the good work!! I've downloaded the snapshot but I cannot use it ... I've manually changed the hobbitnk.cfg file to put an host that exists in my configuration but it seems that nkedit cannot get the configuration (it didn't find that host/service) and if I try to insert anything i get an Internal Error from apache. In the error_log i get Fri Jan 27 11:57:54 2006] [error] [client 10.90.24.45] Premature end of script headers: hobbit-nkedit.sh, referer: http://itromlx10/hobbit-seccgi/hobbit-nkedit.sh?HOSTNAME=itromlx10&SERVI CE=http&Find=Search&PRIORITY=1&GROUP=&NKWKDAYS=*&NKSTARTHOUR=0000&NKENDH OUR=2400&start-day=0&start-mon=0&start-yr=0&end-day=0&end-mon=0&end-yr=0 &EXTRA=&NKEDITADDCLONES= I've also one question (and possibly one request if it can be implemented)... It's possible to create different configuration file for different group ? Here I've some groups that are monitoring various servers/services (windows server, network equipment, Web Logic Server and Apache Web Server) and it can be really nice to have them go to different NK Page (configuration) each one monitoring the objects they really need to check.... I'm also trying to create a comprehensive hobbit configuration program that can edit all the files in the hobbit configuration (hosts, alert, client, services, graphs, etc.). My unix programming skill are a bit old so I'm developing it in .NET for windows (also because some of the user here that can need to modify the configuration are windows people that know nothing about telnet or ssh or vi :P). The interface will be a MDI interface with one child-window for each configuration file and it will be simple to include in the program new configuration file of others addon (as the snmp or bb-msgs and so on). At the moment I'm trying to understand how to get/put the file on the hobbit machine and the simplest one is via ftp... I think I can get the file also via the bbmessage.cgi but then I've to get via bbmessage.cgi and put with ftp ... Continue the good work! Thanks Francesco

1 0

out-of-inodes hobbit conn error for all 300 systems
by iqbala-hobbit＠qwestip.net 27 Jan '06

27 Jan '06

Hi All I am running solaris 9. My hobbit is running off of /export/home/hobbit dir. The `df -i` shows (root)@hobbit:/export/home/hobbit/data$ /usr/local/bin/df -i Filesystem Inodes IUsed IFree IUse% Mounted on /dev/md/dsk/d0 494592 7665 486927 2% / /dev/md/dsk/d40 504000 63200 440800 13% /usr /dev/md/dsk/d20 494592 8052 486540 2% /var swap 98024 112 97912 1% /var/run swap 98024 112 97912 1% /tmp /dev/md/dsk/d50 504000 16322 487678 4% /opt /dev/md/dsk/d30 1005696 1005005 691 100% /export/home But still have enough disk space per `df -k` (root)@bigbrother:/export/home/hobbit/data$ /usr/local/bin/df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/md/dsk/d0 986131 114371 812593 13% / /dev/md/dsk/d40 2033103 1281660 690450 65% /usr /dev/md/dsk/d20 986131 153015 773949 17% /var swap 2705592 24 2705568 1% /var/run swap 2706504 936 2705568 1% /tmp /dev/md/dsk/d50 2033103 850773 1121337 44% /opt /dev/md/dsk/d30 8261393 4437625 3741155 55% /export/home Wish I had the filesystem built optimized for lots of inodes with small files instead. But it is too late. As far as resources on this system I also have 674M actual and 2642M swap memory left. Either I think of using these resources, like may be creating some RAM disks (Henrik had a tip on that, cant find it) or get rid of `hist' folder which has about 4000 files in it. I really like to keep the `hist' files for report and going back to check for old events. Anyone can think of any suggestion to overcome this predicament? Thanks -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu "..there are two kinds of people: those who work and those who take the credit...try to be in the first group;...less competition there." - Indira Gandhi

5 8

Rebuilt server
by lars.ebeling＠leopg9.no-ip.org 27 Jan '06

27 Jan '06

Hi, I was very silly, I thought that Windows didn't write to the files when using ntabackup. I have no working tapestation on my HP-UXserver, so I installed Samba and used ntbackup to backup my server. But Windows change a lot of long filenames. At least is that what I think happened. So almost everything stopped working. But Samba was still running so I started to copy selected directories and config files to Windows. Since I think my mailserver is most critical I installed it on another HP-UXserver and got it to work Now I have reinstalled HP_UX and all needed libraries, compilers and other software. I hope. That's the background. I copied my Hobbit home directory back, and installed a new snapshot. After some trial and error almost everything works. All datafiles are updated. Main view, All non green and NK view seems to work. But when I click on History button I get: Cannot open history file. And no graphs are created. It seems like the data-directory isn't accessible. Regards Lars Ebeling http://leopg9.no-ip.org Hobbithobbyist "Never run after your own hat. Others will be delighted to do it; why spoil their fun?" -- Mark Twain

2 2

[hobbit] Latest Snapshot
by fduranti＠q8.it 27 Jan '06

27 Jan '06

Hi Henrik, Thanks for the good work!! I've downloaded the snapshot but I cannot use it ... I've manually changed the hobbitnk.cfg file to put an host that exists in my configuration but it seems that nkedit cannot get the configuration (it didn't find that host/service) and if I try to insert anything i get an Internal Error from apache. In the error_log i get Fri Jan 27 11:57:54 2006] [error] [client 10.90.24.45] Premature end of script headers: hobbit-nkedit.sh, referer: http://itromlx10/hobbit-seccgi/hobbit-nkedit.sh?HOSTNAME=itromlx10&SERVICE=… I've also one question (and possibly one request if it can be implemented)... It's possible to create different configuration file for different group ? Here I've some groups that are monitoring various servers/services (windows server, network equipment, Web Logic Server and Apache Web Server) and it can be really nice to have them go to different NK Page (configuration) each one monitoring the objects they really need to check.... I'm also trying to create a comprehensive hobbit configuration program that can edit all the files in the hobbit configuration (hosts, alert, client, services, graphs, etc.). My unix programming skill are a bit old so I'm developing it in .NET for windows (also because some of the user here that can need to modify the configuration are windows people that know nothing about telnet or ssh or vi :P). The interface will be a MDI interface with one child-window for each configuration file and it will be simple to include in the program new configuration file of others addon (as the snmp or bb-msgs and so on). At the moment I'm trying to understand how to get/put the file on the hobbit machine and the simplest one is via ftp... I think I can get the file also via the bbmessage.cgi but then I've to get via bbmessage.cgi and put with ftp ... Continue the good work! Thanks Francesco -----Messaggio originale----- Da: Henrik Stoerner [mailto:henrik at hswn.dk] Inviato: giovedì 26 gennaio 2006 21.46 A: hobbit at hswn.dk Oggetto: Re: [hobbit] Latest Snapshot On Thu, Jan 26, 2006 at 12:04:16PM -0500, Michael Dunne wrote: > I downloaded the latest snapshot and am curious if there is any > documentation for the new "Critical Systems" enhancements. The new > features look perfect for our organization and I'd love to start testing. Docs are usually the last thing that gets done :-) Fortunately, this is one of those things that doesn't need much documentation just to use. There are two new CGI's - they both install automatically when you run "make install". The "hobbit-nkview.sh" provides a view of the current critical systems, as defined in a new configuration file hobbit-nkview.cfg; the old "NK" tags in the bb-hosts file have been deprecated since they cannot hold all the info that is needed. So this is a replacement for the bbnk.html page that Hobbit inherited from the bbgen tool. The new view is dynamic - so it's always up-to-date; it allows you to prioritize systems so the most important ones are listed first; it will automatically hide alerts that are older than some threshold so a "noisy" system doesn't clutter the view of the operators; and it allows the operator to acknowledge alerts - both to get them off his display, and to inform others that the alert has been recognized and some action has been taken. The acknowledgement feature is going to be used a lot more in the future; it is designed to allow acknowledgments to be sent by different groups to allow for alerts that can escalate to e.g. higher-level management regardless of any acks from a technician. In my production setup, the configuration of what systems and alerts are "critical" is handled by a separate group who wants some say about what can and cannot go on their monitor (e.g. they won't accept responsibility for monitoring stuff until the operational procedures have been documented etc.); this is a group of people who are very much "point-and-click" types. So hand-editing text configuration files is out of the question. Hence, a web-interface to configuring the critical systems view was built; this is the "hobbit-nkedit.sh" CGI. So by using this tool, you can configure the "critical systems" view with what status messages show up there; what their priority should be; what times of the day/week they are monitored; any special instructions the operators should see when there is an alert; and for systems that are going into or out of service you can even define the date where this happens and the monitoring will automatically start or stop at the right time. For groups of hosts with identical monitoring setups, you can define a template for how they are configured on the "critical systems" view, and then "clone" this template to all of the hosts that should use the same template. So you only need to define the monitoring once. To use this, you basically build the current snapshot, then do a "make install" on top of your current setup. You will need to add a couple of menu items to your ~hobbit/server/www/menu/menu_items.js file, to get the links to these two new CGI's; see the default file that gets built in hobbitd/wwwfiles/menu/menu_items.js when you build Hobbit. In the "Views" menu, it is the "Critical systems" item, and in the "Administration" menu it is the "Edit critical systems" item. Note that the hobbit-nkedit.sh wrapper for the CGI with the current snapshot gets installed in the public CGI directory; it should go in the secured (password-protected) CGI directory. This has been fixed for the next snapshot. Another thing you must do it to arrange for some way of allowing the hobbit-nkedit CGI to update the hobbit-nkview.cfg file. The way it is done, this CGI (which runs with your web-server userid) must have access to update the ~hobbit/server/etc/ directory. The easiest way of doing that is to make the program "suid hobbit". So: chown hobbit ~hobbit/server/bin/hobbit-nkedit.cgi chmod u+s ~hobbit/server/bin/hobbit-nkedit.cgi I haven't yet decided if that is the best way of doing it; I have a bad feeling about making CGI programs suid, but this may be the one occasion where it is needed. Feedback on this is welcome. I'm actually doing a presentation tomorrow for our 24x7 monitoring group, showing them how these new features work. It has been very much designed based on input from them, so hopefully it should work in a way that is useful to the operators. Once I get some confirmation that there are no serious bugs in the new tools, I'll put it into a proper release version - probably sometime in February. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

extermanl temperature sensor
by rmcgraw＠purdue.edu 27 Jan '06

27 Jan '06

When I was looking at the hobbit web page a while back, I thought I remember seeing an advertisement for an external temperature sensor device that was network ready and not to expensive. I cannot see this any more or maybe I am mistaken. Can anyone point me to a unit that is small, easy to mount and not to expensive and that you have used and can recommend. Thanks Robert _____________________________________________________________________ Robert P. McGraw, Jr. Manager, Computer System EMAIL: rmcgraw at purdue.edu Purdue University ROOM: MATH-807 Department of Mathematics PHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067

3 2

Hobbit Communication Protocol
by jonescr＠cisco.com 26 Jan '06

26 Jan '06

Henrik, I was thinking of writing a perl implementation of a mail-ack gateway for Hobbit. I know bb-mailack.sh exists, but it has to be modified to work with hobbit, and I would also like to have the ability to not only ack an alert but to enable/disable hosts via email as well. That being said, I also would like to have my script totally self contained, and not rely on any external binaries or having to source external shell environments. So I was wondering if you had documented anywhere the protocol hobbit uses to communicate - in other words, what I need to do to emulate "./bb hobbitserver hobbitcommand". I can probably figure it out myself by going through the source, but I have a feeling you have it documented, or that its something simple enough to reply with :-) The end result will be a hobbit-mailack.pl script that will be a standalone mail gateway for hobbit which can be called from .forward, /etc/aliases, .qmail, or whatever other method ones MTA might provide for piping email through a script. Options I intend to include are: * Logging of the gateway usage * Ability to ack an alert * Ability to enable or disable a host * Simple configuration Speaking of this, I think it would be helpful if when a service is in alert status, that there was an easy way to ack the alert from the web page *without knowing what the alert ID is*. Something like an "acknowledge *this* alert" (w/ same password protection as admin area) button right smack next to the history button :-) People that I am teaching to use hobbit have commented that they find it inefficient to have to go through their email to find the last alert ID for a service, and cut and paste it into hobbit to ack an alert. -Charles

2 1

temperature.sh I am almost their. PROBLEM connect to bbd failed.
by rmcgraw＠purdue.edu 26 Jan '06

26 Jan '06

I have installed temperature.sh in ext and made a few fixes. 1) temperature.sh seems to need nawk and not awk so I had to set the in the .sh file. 2) It has to pass the name temperature and not temp. This took me a little time to figure out. On the client I entered the following in the clientlaunch.cfg file [temperature] ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg CMD $HOBBITCLIENTHOME/ext/temperature.sh LOGFILE $HOBBITCLIENTHOME/logs/temperature.log INTERVAL 5m This seems to be work just fine. I get the temperatue column and trends. Now on the server I modified the server hobbitlaunch.cfg with the following ## # run dead-cat temperature.sh # [temperature] ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg CMD $HOBBITCLIENTHOME/ext/temperature.sh LOGFILE $HOBBITCLIENTHOME/logs/temperature.log INTERVAL 5m In my client directory temperature.logs I get the following when I start hobbit. *****Sun-Fire-280R*********** 2006-01-26 14:29:43 connect to bbd failed - Connection refused 2006-01-26 14:29:43 Whoops ! bb failed to send message - Connection failed *****Sun-Fire-280R*********** *****Sun-Fire-280R*********** Maybe this happens at the beginning of the start when temperature.sh cannot contact bbd because it is not up yet. _____________________________________________________________________ Robert P. McGraw, Jr. Manager, Computer System EMAIL: rmcgraw at purdue.edu Purdue University ROOM: MATH-807 Department of Mathematics PHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067

1 0

hobbitd_rrd crash
by tlp-hobbit＠holme-pedersen.dk 26 Jan '06

26 Jan '06

I have notiched that my graphs are not updated and thus I found a lot of core files. Is this a filesystem size error ? -bash-2.05b$ file core.9965 core.9965: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV), SVR4-style, from 'hobbitd_rrd' -bash-2.05b$ gdb bin/hobbitd_rrd core.9965 GNU gdb Red Hat Linux (6.1post-1.20040607.52rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1". Core was generated by `hobbitd_rrd --rrddir=/hobbit/hobbit/data/rrd'. Program terminated with signal 25, File size limit exceeded. Reading symbols from /usr/lib/librrd.so.0...done. Loaded symbols for /usr/lib/librrd.so.0 Reading symbols from /usr/lib/libpng12.so.0...done. Loaded symbols for /usr/lib/libpng12.so.0 Reading symbols from /lib/libpcre.so.0...done. Loaded symbols for /lib/libpcre.so.0 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/tls/libm.so.6...done. Loaded symbols for /lib/tls/libm.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 0x0029bf8e in __write_nocancel () from /lib/tls/libc.so.6 (gdb) bt #0 0x0029bf8e in __write_nocancel () from /lib/tls/libc.so.6 #1 0x00238b3f in _IO_new_file_write () from /lib/tls/libc.so.6 #2 0x00237ade in new_do_write () from /lib/tls/libc.so.6 #3 0x00238ccf in _IO_new_file_xsputn () from /lib/tls/libc.so.6 #4 0x002156fe in buffered_vfprintf () from /lib/tls/libc.so.6 #5 0x002113d9 in vfprintf () from /lib/tls/libc.so.6 #6 0x00219f7f in fprintf () from /lib/tls/libc.so.6 #7 0x080512a7 in errprintf (fmt=0x8059554 "RRD error updating %s from %s: %s\n") at errormsg.c:47 #8 0x0804a824 in create_and_update_rrd (hostname=0xb7563038 "comclor04.lu.mach.com", fn=0xffffffff <Address 0xffffffff out of bounds>, creparams=0x805e200, template=0x92794b0 "udpInDatagrams:udpOutDatagrams:udpInErrors:tcpActiveOpens:tcpPassiveOpens:tcpAttemptFails:tcpEstabResets:tcpCurrEstab:tcpOutDataBytes:tcpInInorderBytes:tcpInUnorderBytes:tcpRetransBytes:tcpOutDataPack"...) at do_rrd.c:176 #9 0x08050679 in update_rrd (hostname=0xb7563038 "comclor04.lu.mach.com", testname=0xb756304e "netstat", msg=0xb756307f "status comclor04,lu,mach,com.netstat green Wed Jan 25 22:22:57 CET 2006 <netstat>\nsolaris\n\nRAWIP\n\trawipInDatagrams =5006345\trawipInErrors = 0\n\trawipInCksumErrs = 0\trawipOutDatagra"..., tstamp=1138224178, sender=0xffffffe5 <Address 0xffffffe5 out of bounds>, ldef=0xffffffe5) at do_rrd.c:293 #10 0x08049cbc in main (argc=-27, argv=0xbfff93e4) at hobbitd_rrd.c:199

1 1

Network test time-outs
by JamesHorwath＠glic.com 26 Jan '06

26 Jan '06

I am having an issue with Hobbit network tests. There are times when network or (client) system congestion causes a network test such as SSH to fail. The service is not down just slow, I receive a clear almost immediately after I receive the down. To keep my sanity I need the network value upped. In BB I was able to correct this by setting the variables BBNETTIMER[1-3] to higher values. I wasn't able to find such a beast in Hobbit. I opened the file bbtest-net.c and found the int timeout=10; /* The timeout (seconds) for all TCP-tests */ and int extcmdtimeout = 30; I do not use ICMP as a connectivity test, I use ssh for up/down, to me that is more realistic. I searched the mail archives and didn't find anything. Can I up the values on the time-out to make the test not fail so quickly? Or anybody else have any suggestions on how to work around the issue. Please advise, Jim ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

3 2