March 2008 - Xymon - Xymon mailman web

SunOS does not give back informations to the Hobbit Server
by linuxmasterjedi＠free.fr 31 Mar '08

31 Mar '08

Hi, I have a problem with many Sun/Os Solaris" clients : they don't send any information to the server hobbit. So far, the client compiled OK, run and generate good log files on the client in ~hobbit/client/tmp/msg.hostX.txt I don't have any entries on the server logs nor data. I dropped the machine to force the renewal : nothing. I expected a network issue like wrong hostname or so but seems alright. I've installed the Hobbit client on machine on the same network (Linux/AIX/Windows), they ALL works, except the SunOS ones :-/ Any clue ? Thanks by advance Regards

6 11

FW: [hobbit] tooltips, current snapshot 15-Jan-08
by david.gore＠verizonbusiness.com 31 Mar '08

31 Mar '08

Still hoping this could be fixed soon? Having to scroll the bb2 page for hosts with long descriptions like contact and vendor info is a pain. I still do not really understand why this is only a problem on the bb2 page. ~David _____ From: Gore, David W (David) [mailto:david.gore at verizonbusiness.com] Sent: Thursday, November 15, 2007 14:20 To: hobbit at hswn.dk Subject: RE: [hobbit] tooltips In regards to the hobbit snapshot: They say a picture is worth a thousand words. If the list server allows it, I have attached an image of my bb2.html page. bb2 is our primary view and, as you can see from the image, tooltips pushes all the hobbit columns out of the view to the right. Tooltips are the text you see when you hover over items like the smiley's in hobbit? I suppose it has always worked well for the little smiley gif icons, just not so well for the bb2 host links where it seems to add the tooltip to the hostname rather than create floating text. ~David _____ From: Johann Eggers [mailto:Johann.Eggers at teleatlas.com] Sent: Thursday, November 15, 2007 13:11 To: hobbit at hswn.dk Subject: [hobbit] tooltips Re: the tooltips issue: Isn't it useful to make this an option whether they should be displayed on the bb2 page or not? I really like the tooltips but I agree with David that they will clutter your bb2 page so that this page is not longer usable. Johann _____ From: Gore, David W (David) [mailto:david.gore at verizonbusiness.com] Sent: Donnerstag, 15. November 2007 13:47 To: hobbit at hswn.dk Subject: RE: [hobbit] stale alerts I am running 4.2.0 with the allinone patch. I do have a suspicion that external server side scripts could contribute to the stale alerts. In my case, specifically a script that ssh's to a remote host and executes a shell script on the remote host. It may also happen in conjunction with these errors in the logs, which someone else recently reported: page.log.1:2007-11-05 16:20:53 hobbitd_alert: Got message 14425, expected 14424 As soon as the issue with tooltips is resolved, I will be moving our primary hobbit to the snapshot release. The snapshot, seems much better, but with tooltips and host descriptions pushing our display to the far right of the screen we cannot use it right now. ~David

4 12

BBALPHAMSG for service 'proc', 'disk', etc no line feeds
by waa-hobbit＠revpol.com 30 Mar '08

30 Mar '08

Is there a way to get hobbit to send the BBALPHAMSG variable formatted with line feeds so it appears in an email the way it does on the web page? I guess what I am trying to say is that would be much easier to read this in an email report: --[snip]-- server.domain.tld:procs yellow [584104] yellow Sun Mar 30 10:53:38 EDT 2008 - Processes NOT ok green arkeiad (found 1, req. between 1 and 1) green ntpd (found 1, req. between 1 and 1) green apache2 (found 8, req. 1 or more) green cron (found 1, req. 1 or more) red syslog-ng (found 1, req. 1 or more) green sshd (found 3, req. 1 or more) green slapd (found 1, req. 1 or more) PID TTY STAT TIME COMMAND 1 ? S 0:01 init [3] 2 ? S 0:00 [migration/0] 3 ? SN 0:00 [ksoftirqd/0] 4 ? S 0:00 [migration/1] --[snip]-- as opposed to what I am seeing for a proc report in an email: --[snip]-- server.domain.tld:procs yellow [584104] yellow Sun Mar 30 10:53:38 EDT 2008 - Processes NOT ok &green arkeiad (found 1, req. between 1 and 1) &yellow ntpd (found 0, req. between 1 and 1) &green apache2 (found 8, req. 1 or more) &green cron (found 1, req. 1 or more) &green syslog-ng (found 1, req. 1 or more) &green sshd (found 3, req. 1 or more) &green slapd (found 1, req. 1 or more) PID PPID USER STARTED S PRI %CPU TIME %MEM RSZ VSZ CMD 1 0 root Feb 13 S 23 0.0 00:00:01 0.0 552 1600 init [3] 2 1 root Feb 13 S 139 0.0 00:00:00 0.0 0 0 [migration/0] 3 1 root Feb 13 S 5 0.0 00:00:00 0.0 0 0 [ksoftirqd/0] 4 1 root Feb 13 S 139 0.0 00:00:00 0.0 0 0 [migration/1] --[snip]-- I see similarly unformatted reports for disk partitions. Since there is nothing really to queue off of in the globbed together BBALPHAMSG variable, it is also impossible for me to use sed/awk to manipulate it and format it before emailing.. Any thoughts or comments? Did I miss something obvious? Thanks! -- Bill Arlofski Reverse Polarity, LLC

3 2

2 hobbit servers on the same hardware
by stef.coene＠docum.org 30 Mar '08

30 Mar '08

Hi, Is is possible to run 2 hobbit servers on the same server ? I need a hobbit server for our SLA servers and an other hobbit server for the rest. And I want to run in on the same server. I copied the server directory and changed all the needed files. Each hobbit server is listening on an other port. However, the client information is ending up on1 hobbit server. I think this has to with the way the hobbit daemons are communicating with each other. Stef

6 21

[hobbit] 2 hobbit servers on the same hardware
by alevin＠reveredata.com 29 Mar '08

29 Mar '08

Actually it depends on the model of virtualiziation For example in case of using FreeBSD jails or Solaris zones you won't have any problems with dedicating memory/cpu or running separate instant of the OS ... -- Alex -----Original Message----- From: Phil Wild <philwild at gmail.com> Sent: Saturday, March 29, 2008 8:49 AM To: hobbit at hswn.dk <hobbit at hswn.dk> Subject: Re: [hobbit] 2 hobbit servers on the same hardware Actually, this is where I started.... But it seems such a shame to tie up a fixed amount of memory, disk etc and have the extra burden of another copy of OS as well as the bridged network etc just to run another instance of an application. Unix is a wonderful OS and is so good at scaling I just don't see this as a good use of virtualisation (IMHO) I have gone down the path of recompiling for the second install. There were only about seven entries in the Makefile to modify from memory (not including the port number) Cheers Phil On 30/03/2008, Josh Luthman <josh at imaginenetworksllc.com> wrote: > > I would save hours of time by using a virtual machine. > > > > > On 3/29/08, Phil Wild <philwild at gmail.com> wrote: > > Hi, > > > > Has anyone actually tried this? > > > > I started mucking around with it today but I don't believe it is as easy > as > > it sounds. The original directory "/opt/hobbit/server" seems to be > hardcoded > > into lots and lots of files, including compiled into binaries. I don't > see > > how cloning the hobbit home directory would give you a reliable > > configuration. > > > > I am guessing that the only way to do this would be to reconfigure, > make, > > make install. This would not be my preference but if there is no other > > way... > > > > Comments? > > > > Phil > > > > On 08/03/2008, Larry Barber <lebarber at gmail.com> wrote: > > > > > > All you need to do is give each instance's hobbitd daemon a unique > port to > > > listen to and then make sure everything is talking to the right port > and > > > putting files in the correct directories. Probably the easiest way to > do > > > this is to just clone the Hobbit home directory, then change BBPORT > and > > > BBHOME for the clone. > > > > > > Thanks, > > > Larry Barber > > > > > > On Fri, Mar 7, 2008 at 11:51 AM, Stef Coene <stef.coene at docum.org> > wrote: > > > > > > > Hi, > > > > > > > > Is is possible to run 2 hobbit servers on the same server ? I need > a > > > > hobbit > > > > server for our SLA servers and an other hobbit server for the rest. > > > > And I want to run in on the same server. I copied the server > directory > > > > and > > > > changed all the needed files. Each hobbit server is listening on an > > > > other > > > > port. > > > > > > > > However, the client information is ending up on1 hobbit server. I > think > > > > this > > > > has to with the way the hobbit daemons are communicating with each > > > > other. > > > > > > > > > > > > Stef > > > > > > > > To unsubscribe from the hobbit list, send an e-mail to > > > > hobbit-unsubscribe at hswn.dk > > > > > > > > > > > > > > > > > > > > > -- > > Tel: 0400 466 952 > > Fax: 0433 123 226 > > email: philwild at gmail.com > > > > > > -- > Josh Luthman > Office: 937-552-2340 > Direct: 937-552-2343 > 1100 Wayne St > Suite 1337 > Troy, OH 45373 > > Those who don't understand UNIX are condemned to reinvent it, poorly. > --- Henry Spencer > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > > > -- Tel: 0400 466 952 Fax: 0433 123 226 email: philwild at gmail.com

2 1

Using hobbit's larrd with Big Brother
by Jason.Breitman＠blackrock.com 29 Mar '08

29 Mar '08

I am running Big Brother version 3.3. My company appreciates the support from Quest which is why we have not converted to Hobbit. I was wondering if there is a way to use the portion of hobbit that generates the larrd graphs with Big Brother. We are having performance issues when running larrd on the DISPLAY Server and I know that the Hobbit Version will help us. Jason Breitman jason.breitman at blackrock.com THE INFORMATION CONTAINED IN THIS MESSAGE AND ANY ATTACHMENT MAY BE PRIVILEGED, CONFIDENTIAL, PROPRIETARY OR OTHERWISE PROTECTED FROM DISCLOSURE. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, copying or use of this message and any attachment is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and permanently delete it from your computer and destroy any printout thereof.

4 3

Hobbitd_client on server is crashing
by Martin.Ward＠colt.net 29 Mar '08

29 Mar '08

Hi All, I am running Hobbit 4.2.0 on a number of Sun servers and recently the hobbitd_client daemon has been crashing and leaving core files around. I ran gdb according to http://www.hswn.dk/hobbit/help/known-issues.html#bugreport and it came back with: hobbit at hbt0:/opt/hobbit/server>gdb bin/hobbitd_client tmp/core GNU gdb 6.6 ... Core was generated by `hobbitd_client'. Program terminated with signal 6, Aborted. #0 0xfee25687 in _lwp_kill () from /lib/libc.so.1 (gdb) bt #0 0xfee25687 in _lwp_kill () from /lib/libc.so.1 #1 0xfee22dee in thr_kill () from /lib/libc.so.1 #2 0xfedd11bb in raise () from /lib/libc.so.1 #3 0xfedb15d9 in abort () from /lib/libc.so.1 #4 0x08067172 in sigsegv_handler (signum=11) at sig.c:57 #5 0xfee24a4f in __sighndlr () from /lib/libc.so.1 #6 0xfee1ae72 in call_user_handler () from /lib/libc.so.1 #7 <signal handler called> #8 0x08063ef9 in get_ostype (osname=0x806c28d "") at misc.c:43 #9 0x0805b320 in main (argc=1, argv=0x80464ac) at hobbitd_client.c:1766 The really bizarre thing is that it only crashes when one particular machine connects to it and sends data. Other Hobbit clients connect in and transfer thir core data over with no issues, it's only this one Sun Ultra 60 that causes a problem. Any ideas where I might start looking? |\/|artin -- Martin Ward Network Systems Operations Specialist DDI: +44 (0) 20 7863 5218 Fax: +44 (0) 20 7863 5610 Mob: +44 (0) 7971 97 77 21 www.colt.net Data | Voice | Managed Services Help reduce your carbon footprint | Think before you print COLT Telecommunications, Beaufort House, 15 St Botolph Street, London, EC3A 7QN UK Registered in England and Wales, registered number 02452736, VAT number GB 645 4205 50 ************************************************************************************* The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing security at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and COLT does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of COLT Telecommunications, its subsidiaries or affiliates ("COLT") and any other party by email Communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900.

2 1

F5 BigIP [Devmon] Response to Large or cant Fragment or configuration?
by wrichardson＠llbean.com 28 Mar '08

28 Mar '08

I sent this to the devmon mailing list and thought it might be worth trying the hobbit list to see if anyone was using Hobbit and or Devmon to monitor F5's. Any thoughts/help? -----Original Message----- From: Bill Richardson Sent: Wednesday, March 26, 2008 7:46 AM To: 'Buchan Milne' Cc: devmon-support at lists.sourceforge.net Subject: RE: [Devmon] Response to Large or cant Fragment or configuration? Thanks for your thoughts... Just for fun I add a few of our 4500 Cisco switches that are fully populated and Devmon is working without any issues. I then took another set of bigip F5's that was working with Devmon and increased the number of Pool members one at a time. "The F5's that worked had 80 Pool memebers" When I got to 97 pool members thats when I see "No SNMP data found for PoolMemberStatusReason" in the Devmon log. I delete one of the pool members Devmon starts to work again. "The F5's that Devmon doesn't work has 128 pool members" So this is a test for anyone that is using Devmon to monitor bigip's is to see how many pool members they have. Seems like the braking point is around the 97 mark. I have tcpdumps of a good working "96 members" and the failure "97 members" I could send you but I would like to sanitize them first "some how?" -----Original Message----- From: Buchan Milne [mailto:bgmilne at staff.telkomsa.net] Sent: Tuesday, March 25, 2008 9:24 AM To: devmon-support at lists.sourceforge.net Cc: Bill Richardson Subject: Re: [Devmon] Response to Large or cant Fragment or configuration? On Tuesday 25 March 2008 14:24:03 Bill Richardson wrote: > I'm using devmon to monitor some bigip F5. When devmon is first > started "the first poll" all works. During every poll after on one of > my F5's under the pool "PoolMemberStatusReason" is not being populated > with data. "Agian on the first poll it is populted" > > Looking in the devmon.log is this this message over and over every > minute: > "No SNMP data found for PoolMemberStatusReason on bigip1500" > > Starting devmon with the "-f " I see this starting after the 1st poll > "First poll is fine" > ---------------------------------------------------------------------- > -- > --------------------- > ./devmon -f > > SNMP Error: > Error decoding response PDU: > Expected length 9855, got 7996 > %{%i%s%*{%i%i%i%{%@ > ^ > SNMPv2c_Session (remote host: "192.168.1.10" [192.168.1.10].161) > community: "MyPublic" > request ID: -1368482924 > PDU bufsize: 8000 bytes > timeout: 5s > retries: 3 > backoff: 1) > at /usr/local/devmon-0.3.0-rc1/modules/dm_snmp.pm line 540 Use of > uninitialized value in string ne at > /usr/lib/perl5/site_perl/5.8.8/SNMP_Session.pm line 871, <$__ANONIO__> > line 16. > ---------------------------------------------------------------------- > -- > ------------------------ > > Based on this I upgraded from SNMP_Session 1.08 to 1.12 and still see > the same issue. > > I did some tcpdumps and was thinking it may have been the size of the > response.... > > Looking at the dumps I noticed that the first poll response in the > trace has a total length of only 988, while the second poll response > has a length of 1514. > > Additionally I noticed that second poll packet has the fragment bit set. > meaning there should be more packets to follow. I wonder if the perl > script is having trouble handling a fragmented packet. This might > have something to do with the message that I'm seeing that was > complaining about the expected length of the packet. > > So, thinking it has something to do with the size of the response, as > a test I went in to the bigip box (the standby box) and I deleted a > bunch of the pools - like half of them or more. Once I did that, I > went in and checked devmon again. Once it refreshed it was able to > load the pool info for this box. This proves that it is definitely > related to the size of the response that the SNMP bulkget is receiving > that is causing the error. > > My guess is that is that the first request is working because of that > MAX REPETITION setting. If you look at that first response packet, it > returns 12 SNMP items. At some point between that first request and > the second one, devmon must be learning that there are really 129 > items here, not 12. Then when it tries on the second attempt to pull > in that many item responses they cause the error. Whether it is the > packet fragmenting that is the cause or something else related to the > size of the response, I'm not sure. > > Any help with this one... I have looked and looked dont see this > reported on the list in the past.. I'm currently running perl-SNMP_Session 1.08, and monitoring some cisco devices with more than 180 items per test. It might be best if you could send captured files from tcpdump or wireshark. However, this looks more like an issue with SNMP_Session, than with devmon. But, I'll try and investigate. Regards, Buchan

2 1

graphing questions
by philwild＠gmail.com 28 Mar '08

28 Mar '08

Hello hobbit users, Three items.... We monitor the size of oracle tables in a similar way as the base disk monitoring is performed. We use a custom script at the back end to create the rrd's as new tablespaces appear (i.e. one tablespace per rrd) The tablespace sizes are graphed in a similar way to the disk graphs as well with the standard hobbit method of dividing the tables into smaller groups (I think it is 7) and charting them on one graph. The issue we have is that some tables are very small (say 100MB and some are large 30GBytes). When these are plotted on the same graph, the plots for the small tables are of little value as they look like zero... I assume the only way to address this would be to plot each table individually. Anyone have any ideas how I can force this? Next.... I want to stack all the tables onto an area chart to show total database growth. However, the first value needs to be area and the rest of the values stack. How can I achieve this when a regex is used to pick up the rrd's? And last, I am toying with the idea of trending growth rate. It would seem that the rrd database would make a good source of historical data for this. Has anyone done something here? One idea I am playing with is if a filesystem breaches a capacity threshold out of hours, but the greatest of the last five minute and 30 minute growth rates would indicate that it is not going to run out of space before 10AM on a business day, then don't send me an sms to wake me up. Thoughts? Phil

2 1

history not updating
by stewartl42＠gmail.com 28 Mar '08

28 Mar '08

Not sure if this is related to my issue about Whoops errors or not, but it seems that the history does not always update correctly when a service goes down. Seems to be the same problem as here... http://www.hswn.dk/hobbiton/2008/03/msg00064.html My history.log shows no issues... Bunch of "Will not update <server.test> - color unchanged" messages. hobbitlaunch.log shows this. 2008-03-26 23:21:05 Whoops ! bb failed to send message - timeout 2008-03-26 23:56:17 Whoops ! bb failed to send message - timeout -- Stewart The revolution will not be televised. The revolution will be no re-run brothers; The revolution will be live.

2 2