June 2006 - Xymon - Xymon mailman web

[hobbit] hobbitd_rrd error
by henrik＠hswn.dk 02 Jun '06

02 Jun '06

On Fri, Mar 31, 2006 at 03:21:20PM +1100, Geoff Steer wrote: > In looking at the current core files with gdb, it seems that that they > all report an error related to sendmail: > > (gdb) bt > #5 0x0804e011 in do_sendmail_rrd ( > hostname=0xb7f6f037 "outrelay1.firstwave.com.au", > testname=0xb7f6f052 "sendmail", > msg=0xbffc5dd0 tstamp=1143771322) > at rrd/do_sendmail.c:127 > #6 0x08050120 in update_rrd ( > hostname=0xb7f6f037 "outrelay1.firstwave.com.au", > testname=0xb7f6f052 "sendmail", > msg=0xb7f6f05b "data outrelay1,firstwave,com,au.sendmail Fri Mar 31 > 13:15:22 EST 2006\nStatistics from Tue Jun 21 10:47:07 2005\n M msgsfr > bytes_from msgsto bytes_to msgsrej msgsdis msgsqur Mailer\n 3 > 25299848"..., > tstamp=1143771322, sender=0x0, ldef=0x0) at do_rrd.c:271 > #7 0x08049e3a in main (argc=0, argv=0xbffca4e4) at hobbitd_rrd.c:199 > > I'm ready to rebuild the server entirely but I'm not convinced that this > will resolve the issue. As I said previously, this set up has been > working fine for months, the problem started for no obvious reason in > early march. I'm not sure what was the cause of these crashes, but I looked for a problem around that piece of code and found one missing check for an error returned by the RRDtool library that could explain this. So I'd like some feedback if the problem still occurs, and if it does whether tomorrows snapshot (generated in a few hours) helps. Regards, Henrik

1 0

[hobbit] another bug report on 4.1.2p1
by henrik＠hswn.dk 02 Jun '06

02 Jun '06

On Fri, Mar 10, 2006 at 03:57:51PM +0100, Olivier Beau wrote: > In bb-hosts, settings an apache tag will an invalid url > (apache=www.toto.com/server-status?auto) causes the whole bbtest-net to fails > and coredump This has been fixed today. Henrik

1 0

FreeBSD client where?
by kolbjorn.barmen＠uninett.no 02 Jun '06

02 Jun '06

I'm looking for a hobbit client package for FreeBSD.. is there such a thing? Or perhaps a tarball for ports? I'm not so much into how FreeBSD does these things, from Gentoo I'm used to having an overlay for my own packages, but I've never heard of such functionality in FreeBSD ports. -- Kolbjørn Barmen UNINETT Driftsenter

2 2

msgs clear
by iain＠uk.colt.net 02 Jun '06

02 Jun '06

Morning people, Hopefully, this has not bee discussed on the list before, but here goes anyway... I have replaced my BB server with a hobbit server. Very nice drop in replacement :) I am still getting my head round the differences, but most issues I have had have been solved. Apart from.. My server is called bb02.colt, running on linux. I have added this entry into the ~/server/etc/client-local.cfg [linux] log:/var/log/messages:10240 ignore identification ignore modprobe trigger sudo and this entry into the ~/server/etc/hobbit-clients.cfg LOG /var/log/messages kernel LOG /var/adm/messages ERROR But I get no msgs status on the hobbit servers web pages e.g. System logs at Thu Jun 1 10:07:18 BST 2006 : No log data available The client did not report any logfile data Both the config files are owned by user hobbit. Can a knowledgable person point out where I may have gone wrong please :) Cheers -- Iain Conochie UNIX System Administrator UK COLT Telecom Units 9-13 Powergate Business Centre, Volt Avenue, London NW10 6PW Tel: +44 (0) 20 7947 1286 Tel: +44 (0) 78 1514 5493 Email: iain.conochie at colt.net www.colt.net

3 4

GRAPHS syntax
by Dominique.Frise＠unil.ch 02 Jun '06

02 Jun '06

Hi, I cannot find a complete syntax information for the GRAPHS setting in hobbitserver.cfg. From hobbitserver.cfg: # This defines which RRD files to include on the "trends" column webpage, # and the order in which they appear. From the man page: GRAPHS List of the RRD databases, that should be shown as a graph on the "trends" column. We noticed the use of two separator (:) like "mrtg::1" Can you please describe the use for this. Dominique UNIL - University of Lausanne

2 1

[hobbit] moving hobbit from Ubuntu to Debian
by dan.simoes＠gmail.com 02 Jun '06

02 Jun '06

Telnet to localhost as well as 127.0.0.1 work, yes. The hobbitdboard returns a lot of data. I've literally been looking at this for hours, I don't know what else is wrong. I even did a completely fresh install, dropped in my bb-hosts, same results. I know bb-hosts is OK since it's working elsewhere (I did change the bbd line of course). On 5/31/06, Henrik Stoerner <henrik at hswn.dk> wrote: > > Can you telnet to 127.0.0.1 port 1984 ? If that times out, then you have > a seriously heavy firewall setup. Something like > iptables -I INPUT 1 -i lo -j ACCEPT > iptables -I OUTPUT 1 -i lo -j ACCEPT > should cancel out any paranoid firewall setups locally on the box, for > the loopback-interface. > > Can you do a "~hobbit/server/bin/bb 127.0.0.1 hobbitdboard" and get any > data back ? > > If you're running all of the network tests locally, I think you can > get away with setting > BBSERVERIP="127.0.0.1" > in your hobbitserver.cfg, and let all of the communications happen on > the loopback-interface. Of course, for clients to work you'll need to > get port 1984 opened up for access from the clients. But loopback should > be ok for just getting the Hobbit server up and running. > > > Henrik > > > On Wed, May 31, 2006 at 02:48:33PM -0700, Dan Simoes wrote: > > I thought of that, eventually, and corrected the error. Same problem. > > localhost is responding on 1984, but the site's firewall is blocking > 1984. > > I don't have any clients connecting at the moment, so I don't imagine > this > > is a problem, but thought I would mention it. > > > > On 5/31/06, Henrik Stoerner <henrik at hswn.dk> wrote: > > > > > >Check the IP-adresses for the Hobbit server in the hobbitserver.cfg > > >file. From your logs, it seems that the query to fetch the current > total > > >status from the Hobbit daemon goes to the wrong server (the old IP, > > >probably), so there is no response and therefore the webpages dont get > > >any hosts listed. > > > > > > > > >Henrik > > > > > >On Wed, May 31, 2006 at 02:41:25PM -0700, Dan Simoes wrote: > > >> I got an account on a new virtual host (debian 3.1 on virtuozzo), and > > >> dropped in a tar of my hobbit dir. > > >> I figured I could just edit bb-hosts and be up and running in > minutes. > > >> Hours later, I'm still not up. > > >> > > >> OK, so I was missing fping, and rrdtool. Hobbit did not like the > older > > >> rrdtool 1.X under debian stable, so I switch to testing, upgraded, > and > > >> recompiled and reinstalled hobbit. > > >> It's still not working, even with a simplified bb-hosts. > > >> > > >> hobbit runs, but the webpage output is devoid of any host data. > > >> I can't figure out what else could be wrong, this should have been a > > >very > > >> simple port. > > >> > > >> Logs show: > > >> > > >> bb--display: > > >> 2006-05-31 17:30:22 Whoops ! bb failed to send message - timeout > > >> 2006-05-31 17:30:22 hobbitd status-board not available > > >> 2006-05-31 17:30:22 WARNING: Runtime 92 longer than BBSLEEP (60) > > >> 2006-05-31 17:31:09 Whoops ! bb failed to send message - timeout > > >> > > >> bb-network: > > >> 2006-05-31 17:30:27 Whoops ! bb failed to send message - timeout > > >> 2006-05-31 17:30:27 WARNING: Runtime 140 longer than time limit (120) > > >> 2006-05-31 17:31:14 Whoops ! bb failed to send message - timeout > > >> > > >> > > >> hobbitclient.log: > > >> 2006-05-31 17:32:47 Whoops ! bb failed to send message - timeout > > >> error parsing /proc/net/snmp: Success > > >> 2006-05-31 17:34:47 Whoops ! bb failed to send message - timeout > > >> error parsing /proc/net/snmp: Success > > > > > >-- > > >Henrik Storner > > > > > >To unsubscribe from the hobbit list, send an e-mail to > > >hobbit-unsubscribe at hswn.dk > > > > > > > > > > > -- > Henrik Storner > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > > >

3 6

[hobbit] moving hobbit from Ubuntu to Debian
by Mike.Rowell＠Rightmove.co.uk 02 Jun '06

02 Jun '06

As another idea, I had a very similar problem over the last few days, same Whoops messages and it turned out to be a problem with the resolv.conf, not sure exactly what the problem with it was as I was in a bit of a rush as I had been trying to figure out the problem all day. I cleared it down to just the nameservers no search or domain parameters and it all started working fine again (This is on Solaris 10) Mike -----Original Message----- From: John Glowacki [mailto:johng at idttechnology.com] Sent: 02 June 2006 05:42 To: hobbit at hswn.dk Subject: Re: [hobbit] moving hobbit from Ubuntu to Debian This probably isn't going to be much help with this problem, but Henrik might find the feed back of some interest. I recently rebooted a Sun server. Later the hobbit client status for the server went purple and I noticed I was getting these errors in the hobbitclient.log on that server: 2006-05-25 06:58:38 Whoops ! bb failed to send message - timeout 2006-05-25 07:03:43 Whoops ! bb failed to send message - timeout 2006-05-25 07:08:47 Whoops ! bb failed to send message - timeout However I was getting one status from the client. It was from an add-on application called SE Tool Kit. It displayed the following status: green Disks 2%busy No activity green Nets No activity green NFS No client NFS/RPC activity green Swap There is a lot of unused swap space green RAM 86%free RAM available green Kmem No worries, mate green CPU CPU idling green Mutex No worries, mate green DNLC No worries, mate green Inode No activity yellow TCP High retransmissions, check TCP patch level From this we were able to determine there was a network problem. Our LAN group got involved and resolved the retransmission errors from their side. Getting back to the original Whoops error. In our case it seems the hobbit client was not getting thru because of the high retransmissions. As for why the addon messages got thru and the rest of the hobbit client messages didn't I will leave that up to Henrik for his thoughts on it. For those who may be interested in setting up SE Tool Kit with the hobbit client for Sun. You can get it from here http://www.sunfreeware.com/setoolkit.html. There is an example file included in the package called bigbrother.se. I renamed it to hobbit.se, updated the path for calling bin/bb and changed the BBHOST variable to BBDISP. Add an entry to clientlaunch.cfg: [se] ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg CMD /opt/RICHPse/bin/se /opt/hobbit/client/hobbit.se LOGFILE $HOBBITCLIENTHOME/logs/hobbitclient.log We also run Orca(http://www.orcaware.com/orca/). It's a performance and trend analysis package. It also uses the SE Tool kit and creates detailed graphs. Here is an what it looks like. http://www.orcaware.com/orca/docs/orcallator.html#TCP_retransmission_dup licate_received_percentage With how fast Hobbit is growing we may someday be able to drop the Orca package. John Dan Simoes wrote: > I appreciate the help. Running the debug you sent, it seems that the box > cannot connect to itself on 1984 using its public IP. > I will see if I can remove the firewall for now (it's not iptables, it's > something the provider gives you). > Thanks again. > > On 5/31/06, Henrik Stoerner <henrik at hswn.dk> wrote: > >> >> One more thing before I go to sleep :-) >> >> Check your hobbitserver.cfg - if you have BBDISP set the same as >> BBSERVERIP. That *is* the default, but you might have changed it. >> It should be >> BBDISP="$BBSERVERIP" >> in hobbitserver.cfg. >> >> >> Regards, >> Henrik >> >> On Thu, Jun 01, 2006 at 12:04:55AM +0200, Henrik Stoerner wrote: >> > Login as the hobbit user, then run >> > ./server/bin/bbcmd bbgen --debug | less >> > A bit into that log is where bbgen tries to fetch the current status >> > from the Hobbit daemon. It looks like this: >> > >> > 2006-06-01 00:02:24 load_state() >> > 2006-06-01 00:02:24 Transport setup is: >> > 2006-06-01 00:02:24 bbdportnumber = 1984 >> > 2006-06-01 00:02:24 bbdispproxyhost = NONE >> > 2006-06-01 00:02:24 bbdispproxyport = 0 >> > 2006-06-01 00:02:24 Recipient listed as '127.0.0.1' >> > 2006-06-01 00:02:24 Standard BB protocol on port 1984 >> > 2006-06-01 00:02:24 Will connect to address 127.0.0.1 port 1984 >> > 2006-06-01 00:02:24 Connect status is 0 >> > 2006-06-01 00:02:24 Sent 118 bytes >> > 2006-06-01 00:02:24 Read 3364 bytes >> > >> > Does it report the right IP-address as the 'Recipient' ? >> > >> > Henrik >> > >> > >> > On Wed, May 31, 2006 at 03:00:19PM -0700, Dan Simoes wrote: >> > > Telnet to localhost as well as 127.0.0.1 work, yes. >> > > The hobbitdboard returns a lot of data. >> > > >> > > I've literally been looking at this for hours, I don't know what else >> is >> > > wrong. >> > > I even did a completely fresh install, dropped in my bb-hosts, same >> results. >> > > I know bb-hosts is OK since it's working elsewhere (I did change the >> bbd >> > > line of course). >> > > >> > > On 5/31/06, Henrik Stoerner <henrik at hswn.dk> wrote: >> > > > >> > > >Can you telnet to 127.0.0.1 port 1984 ? If that times out, then you >> have >> > > >a seriously heavy firewall setup. Something like >> > > > iptables -I INPUT 1 -i lo -j ACCEPT >> > > > iptables -I OUTPUT 1 -i lo -j ACCEPT >> > > >should cancel out any paranoid firewall setups locally on the box, >> for >> > > >the loopback-interface. >> > > > >> > > >Can you do a "~hobbit/server/bin/bb 127.0.0.1 hobbitdboard" and get >> any >> > > >data back ? >> > > > >> > > >If you're running all of the network tests locally, I think you can >> > > >get away with setting >> > > > BBSERVERIP="127.0.0.1" >> > > >in your hobbitserver.cfg, and let all of the communications >> happen on >> > > >the loopback-interface. Of course, for clients to work you'll >> need to >> > > >get port 1984 opened up for access from the clients. But loopback >> should >> > > >be ok for just getting the Hobbit server up and running. >> > > > >> > > > >> > > >Henrik >> > > > >> > > > >> > > >On Wed, May 31, 2006 at 02:48:33PM -0700, Dan Simoes wrote: >> > > >> I thought of that, eventually, and corrected the error. Same >> problem. >> > > >> localhost is responding on 1984, but the site's firewall is >> blocking >> > > >1984. >> > > >> I don't have any clients connecting at the moment, so I don't >> imagine >> > > >this >> > > >> is a problem, but thought I would mention it. >> > > >> >> > > >> On 5/31/06, Henrik Stoerner <henrik at hswn.dk> wrote: >> > > >> > >> > > >> >Check the IP-adresses for the Hobbit server in the >> hobbitserver.cfg >> > > >> >file. From your logs, it seems that the query to fetch the >> current >> > > >total >> > > >> >status from the Hobbit daemon goes to the wrong server (the old >> IP, >> > > >> >probably), so there is no response and therefore the webpages >> dont >> get >> > > >> >any hosts listed. >> > > >> > >> > > >> > >> > > >> >Henrik >> > > >> > >> > > >> >On Wed, May 31, 2006 at 02:41:25PM -0700, Dan Simoes wrote: >> > > >> >> I got an account on a new virtual host (debian 3.1 on >> virtuozzo), and >> > > >> >> dropped in a tar of my hobbit dir. >> > > >> >> I figured I could just edit bb-hosts and be up and running in >> > > >minutes. >> > > >> >> Hours later, I'm still not up. >> > > >> >> >> > > >> >> OK, so I was missing fping, and rrdtool. Hobbit did not like >> the >> > > >older >> > > >> >> rrdtool 1.X under debian stable, so I switch to testing, >> upgraded, >> > > >and >> > > >> >> recompiled and reinstalled hobbit. >> > > >> >> It's still not working, even with a simplified bb-hosts. >> > > >> >> >> > > >> >> hobbit runs, but the webpage output is devoid of any host data. >> > > >> >> I can't figure out what else could be wrong, this should have >> been a >> > > >> >very >> > > >> >> simple port. >> > > >> >> >> > > >> >> Logs show: >> > > >> >> >> > > >> >> bb--display: >> > > >> >> 2006-05-31 17:30:22 Whoops ! bb failed to send message - >> timeout >> > > >> >> 2006-05-31 17:30:22 hobbitd status-board not available >> > > >> >> 2006-05-31 17:30:22 WARNING: Runtime 92 longer than BBSLEEP >> (60) >> > > >> >> 2006-05-31 17:31:09 Whoops ! bb failed to send message - >> timeout >> > > >> >> >> > > >> >> bb-network: >> > > >> >> 2006-05-31 17:30:27 Whoops ! bb failed to send message - >> timeout >> > > >> >> 2006-05-31 17:30:27 WARNING: Runtime 140 longer than time limit >> (120) >> > > >> >> 2006-05-31 17:31:14 Whoops ! bb failed to send message - >> timeout >> > > >> >> >> > > >> >> >> > > >> >> hobbitclient.log: >> > > >> >> 2006-05-31 17:32:47 Whoops ! bb failed to send message - >> timeout >> > > >> >> error parsing /proc/net/snmp: Success >> > > >> >> 2006-05-31 17:34:47 Whoops ! bb failed to send message - >> timeout >> > > >> >> error parsing /proc/net/snmp: Success >> > > >> > >> > > >> >-- >> > > >> >Henrik Storner >> > > >> > >> > > >> >To unsubscribe from the hobbit list, send an e-mail to >> > > >> >hobbit-unsubscribe at hswn.dk >> > > >> > >> > > >> > >> > > >> > >> > > > >> > > >-- >> > > >Henrik Storner >> > > > >> > > >To unsubscribe from the hobbit list, send an e-mail to >> > > >hobbit-unsubscribe at hswn.dk >> > > > >> > > > >> > > > >> > >> > -- >> > Henrik Storner >> > >> > To unsubscribe from the hobbit list, send an e-mail to >> > hobbit-unsubscribe at hswn.dk >> > >> > >> >> -- >> Henrik Storner >> >> To unsubscribe from the hobbit list, send an e-mail to >> hobbit-unsubscribe at hswn.dk >> >> >> > To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk ________________________________________________________________________ This email has been scanned for all viruses by the MessageLabs service. ________________________________________________________________________ This email has been scanned for all viruses by the MessageLabs service. ________________________________________________________________________

1 0

Multiple Displays
by paul.moore＠verizonbusiness.com 02 Jun '06

02 Jun '06

I'm trying to setup a redundant set of servers to be the BBDISPLAYS and a redundant set of servers to be the BBNET pollers. I've managed to setup everything and get it working the way I like except one problem I am having. I've created several scripts to check various items and send status messages to the BBDISPLAY using the $BB cmd. BB="$BBHOME/bin/bb" BBHOME="/usr/local/hobbit/server" BBDISP="0.0.0.0" BBDISPLAYS="172.16.1.50 172.16.2.50" The cmd I'm running looks like this. $BB $BBDISP "status $MACHINE.$COLUMN $COLOR $DATE\t$MSG\n\n"` The result appears that the /bin/bb command will fire off a status but only to the first BBDISPLAY in the list. Is there some other way to make it fire off status to both servers? Paul -=-=-= Pinky, Are You Pondering What I'm Pondering? Well, I think so Brain but if Jimmy cracks corn and no one cares, why does he keep doing it?

4 3

[hobbit] /data/logs directory
by jeffnewman75＠gmail.com 02 Jun '06

02 Jun '06

Henrik, If you have the bbstatus task enabled, does it write the values it recieves to a file, and then the next time data arrives, overwrite the info in the file, or does it append to the file? I would like to capture some data that comes in every 1 or 2 seconds Thanks, Jeff On 5/10/06, Henrik Stoerner <henrik at hswn.dk> wrote: > On Wed, May 10, 2006 at 02:49:39PM -0000, Sigurður Bjarnason wrote: > > > In the old days when I was using the Big Brother monitoring server I > > always got some files to the /data/logs directory. > > That was the Big Brother way of storing data. Hobbit keeps all of the > current status logs in memory - unless you explicitly ask it to store > them on disk by running the hobbitd_filestore module. > > > Since I begun to use Hobbit I have never seen any files there.... > > Witch is causing me some trouble on custom made script I wrote when > > using Big Brother, then I got files to this directory, example > > file1 and file2 and I could compare the two, now ..I just get files to > > /data/hist and histlogs with timestamp in the name and that files I cannot > > use..! > > Either enable the [bbstatus] task in your hobbitlaunch.cfg file - this > causes the data/logs/ directory to be updated with the current status > logs like BB; or change your script to fetch the status messages you > need. I would recommend changing your script ... > > To grab a status log from Hobbit, use the command > bb 127.0.0.1 "hobbitdlog host=HOSTNAME test=COLUMNNAME" > The first line of the output is a summary line, after which comes the > raw status log. > > > Regards, > Henrik > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > > >

2 1

Multiple entries in hobbitgraph.cfg
by rsmrcina＠wi.rr.com 01 Jun '06

01 Jun '06

In the 20060423 alfa there appear to be multiple entries for [procs] in hobbitgraph.cfg. Which one takes precedence? -- Rich Smrcina VM Assist, Inc. Main: (262)392-2026 Cell: (414)491-6001 Ans Service: (360)715-2467 rich.smrcina at vmassist.com Catch the WAVV! http://www.wavv.org WAVV 2007 - Green Bay, WI - May 18-22, 2007

2 2