November 2009 - Xymon - Xymon mailman web

[hobbit] Hobbit: tests suddenly went purple and they did not reappear
by Francesco.Cicolani＠atac.roma.it 01 Dec '09

01 Dec '09

An HTML attachment was scrubbed... URL: <http://lists.xymon.com/pipermail/xymon/attachments/20091130/720e2428/attach…>

2 1

dynamic comment
by bb4＠richter-it.net 30 Nov '09

30 Nov '09

Hi, is it possible to use a dynamic comment in bb-hosts? In Big Brother I patched mkbb2.sh to display a phone number next to the name that changed depending of the time of day. These phone numbers were taken from a separate config file. But with XYmon I only can put a comment to the line in bb-hosts and it is no solution to run a script that changes bb-hosts every time a number has to be changed. I'd like to have something like "COMMENT:$SCRIPTNAME" and this script then is called and gives you back the phone number that is then displayed next to the server name as comment. TIA Torsten

1 0

devmon PID not running.
by am.srini＠gmail.com 30 Nov '09

30 Nov '09

*Hi Guys, Please, Please help me out. I have installed Xymon 4.3.0-0.beta2 on Sun Solaris* *(SunOS bsma06 5.10 Generic_139555-08 sun4u sparc SUNW,Sun-Fire-480R), I had successful installation. I had installed Devmonn-0.3.0. and SNMP_Session-1.13 also to monitor few network devices (Cisco Switches and routers). however when I grep for devmon no PID shows up as shown below: -bash-3.00# ps -ef | grep devmon root 6849 3708 0 14:06:33 pts/1 0:00 grep devmon -bash-3.00# ps -ef | grep Devmon root 6858 3708 0 14:06:42 pts/1 0:00 grep Devmon -bash-3.00# ps -ef | grep DEVMON root 6897 3708 0 14:07:01 pts/1 0:00 grep DEVMON As shown in devmon.cfg # PIDFILE: Sets the full path to the Devmon process ID file. The directory # containing this file should be read/writable by the user running the Devmon # process. (case sensitive) [DEFAULT: /var/run/devmon/devmon.pid] PIDFILE=/var/run/devmon/devmon.pid Please let me know what more details you may require. some info I am providing: I manually changed owner and group of files to root:root, earlier they were 501:501, which I didn't understand. /usr/local/**devmo**n -bash-3.00$ ls -l total 86 -rw-r--r-- 1 root root 836 Nov 30 14:51 @!: -rw-r--r-- 1 root root 9001 Apr 3 2008 CHANGELOG -rw-r--r-- 1 root root 15122 Apr 3 2008 COPYING -rw-r--r-- 1 root root 4935 Apr 3 2008 README -rwxr-xr-x 1 root root 1153 Apr 3 2008 devmon -rw-r--r-- 1 root root 5999 Nov 30 14:08 devmon.cfg drwxr-xr-x 2 root root 512 Apr 3 2008 docs drwxr-xr-x 2 root root 512 Apr 3 2008 extras -rw-r--r-- 1 root root 119 Nov 30 15:01 hosts.db drwxr-xr-x 2 root root 512 Apr 3 2008 modules drwxr-xr-x 40 root root 1024 Apr 3 2008 templates -bash-3.00$ Thanks so much. Srini *

2 1

Hobbit: tests suddenly went purple and they did not reappear
by Francesco.Cicolani＠atac.roma.it 27 Nov '09

27 Nov '09

An HTML attachment was scrubbed... URL: <http://lists.xymon.com/pipermail/xymon/attachments/20091127/003fb2f2/attach…>

2 1

clientupdate The client did not report any logfile data and rrd-data.log error updating
by william.wagner＠ipacc.com 25 Nov '09

25 Nov '09

I am running xymon 4.2.3 on a solaris platform. I am deployed to 300+ clients and all is working. Today I decided to use clientupdate for the first time. The two clients I configured to receive the updates were updated but now the 'msgs' test on both clients returns "No log data available" and "The client did not report any logfile data". All other tests are still working. in a probably unrelated issue, I also see persistent messages in rrd-data.log: 2009-11-25 10:47:16 RRD error updating /export/home/xymon/data/rrd/<hostname>/ifstat.mac.rrd from <IP>: illegal attempt to update using time 1259167636 when last update time is 1259167636 (minimum one second step) any idea what is going on? thanks

1 0

AntiSpam & RBL Hits - Does it Make Sense to Monitor?
by wiskbroom＠hotmail.com 24 Nov '09

24 Nov '09

Hello All; Well, the more you play with Xymon, the more uses you come up with for it. For example, I am responsible for our AV/antispam/Postfix server, on that server we block inbound email based on anything from being on our "do not allow" list, to being on one of two RBL's we use, and of course being identified as either spam or a virus. So where am I going with this? I'd like to know if anyone out there is monitoring something similar, and query you folks to see if this even makes sense for Xymon? One reason why I'd like to add this into Xymon is that we already use too many "tools", and I'd like to either rid us of some of these or condense some of them into just one tool where possible. Of course having Xymon with its rrd graphs would play an important role too since my management would love to see that data in a graphical representation. Thanks all in advance, .vp

4 3

Procs --> output of a process from ps -ef is too long
by camorae＠googlemail.com 24 Nov '09

24 Nov '09

Hello guys, im having some problems to monitor processes with a long output (most of them are java processes). Hobbit seems to cut the output at certain size and i have to look for a string which appears at the end of the output. where can i change this length limit? tia, Cayo

5 9

Optimizing Xymon disk performance (was: Moving RRD processing to another server)
by henrik＠hswn.dk 24 Nov '09

24 Nov '09

Hi Greg, I've taken the liberty of sending this to the Xymon list also, since it is probably of general interest. On Mon, Nov 02, 2009 at 11:39:09AM -0500, shea_greg at emc.com wrote: > > I'm having some trouble trying to figure out how to off-load RRD > processing with the 4.3.0 code. I found hobbitd_locator and that's > part of it, as well as hobbitd_channel, but it's not clear to me > how to setup the master and peer(s). Also how does this affect > the webpage generation? > > From earlier posts to the list, I have a single server running 4.2.0 > with over 70000 RRD files and I'm experiencing serious delays in processing > data and have to restart Hobbit every 15 minutes. One solution I'm aware > of and also Buchan had mentioned is to add more spindles. The standard 4.3.0 beta adds caching of RRD file updates, and this has a significant impact on the I/O load of the server - essentially, it means that hobbitd_rrd caches up to 12 updates (= 1 hour) before it does an actual update of the RRD file. Since the amount of disk I/O is almost identical whether you're doing one data update or 50, this caching eliminates about 90% of your disk I/O on the RRD files. So that would be the simplest solution to implement. I have 90000+ RRD files. I recently did a hardware upgrade of the server, but it isn't anything fancy - just a plain HP DL360 with a set of two 36 GB SCSI disks in hardware RAID-1. I used to off-load the RRD handling to another server, but it is now back on the main Xymon server. The amount of memory used for the cache isn't all that much - about 50 MB on my system. The only downside to this is that shutting down Xymon means all of the cached data must be flushed to disk - and this take a while, 10-15 minutes on my system. Another optimization to eliminate disk I/O is to move the generated webpages to a RAM disk. I have ~hobbit/server/www/ on a ram-disk; the gifs, help, menu, notes, rep and snap sub-directories are symlinks that point to "real" (disk-based) storage. This means all of the webpages that are re-generated once a minute resides on a RAM disk, eliminating all of the disk I/O that rewriting them causes. And since they are regenerated so often, it doesn't matter that they're wiped out when you reboot the server - they are regenerated within a minute after you have Xymon up and running again. But the remote RRD off-loading works, I've used it for more than a year. Here's how to set it up. First, webpage-generation is unchanged, it still happens on the main Xymon server and the fact that the RRD files are stored somewhere else is transparent. The main server runs the hobbitd_locator, which keeps track of where each of the hosts store their RRD files. The RRD server(s) only run hobbitd_rrd, and a webserver. On the main server, add these entries to your hobbitlaunch.cfg: [locator] ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg LOGFILE $BBSERVERLOGS/locator.log NEEDS hobbitd CMD hobbitd_locator --listen=0.0.0.0:9000 [netrrd-status] ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg NEEDS locator CMD hobbitd_channel --channel=status \ --log=$BBSERVERLOGS/netrrd-status.log \ --locator=127.0.0.1 \ --service=rrd [netrrd-data] ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg NEEDS locator CMD hobbitd_channel --channel=data \ --log=$BBSERVERLOGS/netrrd-data.log \ --locator=127.0.0.1 \ --service=rrd The locator listens on port 9000 - it is a UDP based service (like DNS), so you may need to open up some firewalls to reach it. On the RRD offload-servers, you run only the hobbitd_rrd modules with some additional options that tell it to listen for data from a network connection. Here's the hobbitlaunch.cfg entry, assuming your main Xymon server has IP 192.168.1.1 and the RRD off-load server has IP 192.168.1.2: [netrrd-worker] ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg CMD hobbitd_rrd \ --log=$BBSERVERLOGS/netrrd-status.log \ --rrddir=/var/lib/hobbit/rrd \ --locator=192.168.1.1:9000 \ --listen=192.168.1.2:9001 \ --locatorid=192.168.1.2:9001 \ --locatorextra=http://192.168.1.2/hobbit-cgi/ OK, this is a bit complicated - I'll try to explain what these options do. hobbitd_locator needs to know that this RRD-offload-server exists, and what hosts it is handling RRD files for. So hobbitd_rrd must announce itself to the locator - so the "--locator" option tells it how to contact the locator. hobbitd_rrd receives data from the remote hobbitd_channel over a network connection, so the "--listen" option tells it what IP and port-number it will use to listen for incoming connections from hobbitd_channel. The IP/portnumber that hobbitd_rrd listens on may not be the one that hobbitd_channel should use, because the RRD offload server could be hidden behind a NAT firewall or some other network-based address translation might be taking place. So the "--locatorid" option announces the IP+portnumber that hobbitd_channel should use to connect to the hobbitd_rrd service from the outside. Normally there is no NAT'ing, so "--listen" and "--locatorid" are identical. Finally, the "--locatorextra" tells the Xymon web-page tools what URL they should use when generating links to the Xymon graphs. Since the RRD files are no longer stored on the main Xymon server, you cannot access them via the same URL prefix that you use for all of the other Xymon webpages and CGI's - the "--locatorextra" option is used to tell Xymon what the URL is for the graphs. And yes, this means you will need to run a separate webserver on the RRD off-load server. When hobbitd_rrd starts up with these options, it will first contact the locator and tell it "hey, I can handle RRD files - if someone wants to send me some RRD data, they can contact me on 192.168.1.2 port 9001. And please pass this information to anyone who asks for it: http://192.168.1.1/cgi-bin". It then proceeds to scan the RRD directory to determine which hosts it has RRD files stored for, and for each host it then tells the locator "Hi, I am the RRD server on 192.168.1.2:9001, and I have RRD files for host foo.bar.com". After that, it just leans back and waits for someone to connect to it. Over on the main Xymon server, the hobbitd_channel modules are receiving data about RRD updates. Each time a new message arrives, they'll ask the locator "where are the RRD files stored for host abc1.bar.com" ? If the locator knows, then it will respond with the IP:portnumber of the RRD-server handling this host; if it knows that none of the known RRD servers handle this host (i.e. it is a new host) then it will just hand out the IP:portnumber of one of the RRD servers so new hosts can be added. When the hobbitd_channel module is told "send data for foo.bar.com to the RRD server at 192.168.1.2:9001" it will establish a TCP connection to that port (if it doesn't have one open already), and send the data to it. When hobbitd_rrd receives a new connection, it spawns an extra process to handle the connection, which receives the data and then does the actual RRD update. The connections between hobbitd_channel and the RRD offload-server are persistent, so once it is up and running you'll see two connections to your RRD offload server; one for each of the hobbitd_channel instances. The final piece of the puzzle is when you view the detailed status-log on the webpage, and the graph must show up on that page. The hobbitsvc.cgi utility will ask the locator "where are the RRD files for host foo.bar.com?" and get a response that includes the extra data that the locator was asked to pass on to anyone who asked. hobbitsvc.cgi knows that this data is the base of the CGI-URL for the RRD graph CGI, so instead of generating a link to the image URL on the main Xymon webserver, it generates a link that points to the RRD off-load server. The browser contacts the webserver running on the RRD-server, and the image is generated by the RRD-server. I hope that is enough to get you going. Regards, Henrik

4 5

New Xymonton feature
by Galen.Johnson＠sas.com 23 Nov '09

23 Nov '09

Greetings folks I figured while I was focused on updating the Xymonton site, I would add something new. There is now a hidden tag that can be used to collapse a block of, well, whatever. I used it to collapse those long sections of code. I've updated the page templates to include it and editted all the existing pages to add it. Hopefully, it will make some of the pages a bit more readable. While I was at it, I fixed a few pages that made no attempt to use the templates and made some minor modifications to a few others. I'm also looking into some other additions, including captcha...which seems to be sort of working. I may be missing a required module but I'll deal with that later. Remember, until the new dns propagates, you can get to the site at http://www.trantor.org/xymonton as well. =G=

3 2

Devmon causing core dumps
by Vernon.Everett＠woodside.com.au 23 Nov '09

23 Nov '09

Hi all Devmon was causing the hobbitd_rrd module to crash and burn. Now this could be a bug, but it could also be a PEBKAC. I am hoping somebody can assist either way. I added a Cisco 2851 to Hobbit, using devmon. Now here is the possible PEBKAC Since Devmon doesn't have templates for the 2851, I used the template for the Cisco 2811. (Network guru told me they are pretty much the same, except for a few extra bells and whistles on the 2851.) The data for the device started appearing in Hobbit, and all looked good. Devmon even created the rrd files for the new Cisco device. However, the hobbitd_rrd module started core dumping, and the Hobbit server page started displaying red for hobbitd_rrd with the crash detected message. See core data below. Took the new Cisco device out of Hobbit, and cores stopped, and life was good again. Is there a significant enough difference between the 2851 and the 2811 to cause this, or are we looking at a genuine bug? I am leaning towards a bug, because even if the collected data was complete rubbish, should it cause the module to core? Regards Vernon My Linux guy reckons this is the important stuff from the core. uname -a Linux las006 2.6.18-92.1.1.el5 #1 SMP Thu May 22 09:01:47 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux cat /etc/redhat-release Red Hat Enterprise Linux Client release 5.2 (Tikanga) gdb -c core.8550 /usr/lib/hobbit/server/bin/hobbitd_rrd GNU gdb Red Hat Linux (6.5-37.el5_2.1rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db library "/lib64/libthread_db.so.1". Reading symbols from /usr/lib64/librrd.so.2...done. Loaded symbols for /usr/lib64/librrd.so.2 Reading symbols from /usr/lib64/libpng12.so.0...done. Loaded symbols for /usr/lib64/libpng12.so.0 Reading symbols from /lib64/libpcre.so.0...done. Loaded symbols for /lib64/libpcre.so.0 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /usr/lib64/libfreetype.so.6...done. Loaded symbols for /usr/lib64/libfreetype.so.6 Reading symbols from /usr/lib64/libz.so.1...done. Loaded symbols for /usr/lib64/libz.so.1 Reading symbols from /usr/lib64/libart_lgpl_2.so.2...done. Loaded symbols for /usr/lib64/libart_lgpl_2.so.2 Reading symbols from /lib64/libm.so.6...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Core was generated by `hobbitd_rrd --rrddir=/var/lib/hobbit/rrd --debug'. Program terminated with signal 6, Aborted. #0 0x0000003db7a30155 in raise () from /lib64/libc.so.6 (gdb) bt #0 0x0000003db7a30155 in raise () from /lib64/libc.so.6 #1 0x0000003db7a31bf0 in abort () from /lib64/libc.so.6 #2 0x00000000004119f3 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x0000003db7a77ac0 in strcat () from /lib64/libc.so.6 #5 0x000000000040462a in do_devmon_rrd (hostname=0x2ada311e2806 "PERIR205", testname=0x2ada311e280f "if_load", msg=<value optimized out>, tstamp=<value optimized out>) at rrd/do_devmon.c:87 #6 0x000000000040b656 in update_rrd (hostname=0x2ada311e2806 "PERIR205", testname=0x2ada311e280f "if_load", msg=0x2ada311e2842 "status PERIR205.if_load green Fri Oct 31 10:31:39 2008", tstamp=1225416699, sender=<value optimized out>, ldef=0xfeffffffffffff00) at do_rrd.c:372 #7 0x000000000040261d in main (argc=<value optimized out>, argv=0x7fff7a088318) at hobbitd_rrd.c:153 (gdb) NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments.

4 8