February 2006 - Xymon - Xymon mailman web

Hobbit crashes
by christian.maxeiner＠usd.de 01 Feb '06

01 Feb '06

Hi all, I have just migrated from BB to Hobbit. I have run Hobbit beside my old BB installation on port 1985 for testing and configuring. Today I have shut down old BB and switched my Hobbit server and clients back to port 1984. Everything worked fine but an hour later some of the hobbit processes are crashing shortly after restarting hobbit. I am running hobbit 4.1.2p1 on HP-UX 11.11. This is the output of the hobbitlaunch.log file: ... 2006-02-01 16:51:27 hobbitlaunch starting 2006-02-01 16:51:27 Loading tasklist configuration from /users/hobbit4.0//server/etc/hobbitlaunch.cfg 2006-02-01 16:51:27 Loading hostnames 2006-02-01 16:51:27 Loading saved state 2006-02-01 16:51:27 Setting up network listener on 0.0.0.0:1984 2006-02-01 16:51:27 Setting up signal handlers 2006-02-01 16:51:27 Setting up hobbitd channels 2006-02-01 16:51:27 Setting up logfiles 2006-02-01 16:51:34 Task hobbitd terminated by signal 6 2006-02-01 16:51:34 Task bbretest terminated by signal 15 2006-02-01 16:51:34 Task bbdisplay terminated by signal 15 2006-02-01 16:51:34 Task clientdata terminated, status 1 2006-02-01 16:51:34 Task rrddata terminated, status 1 2006-02-01 16:51:34 Task rrdstatus terminated, status 1 2006-02-01 16:51:34 Task bbhistory terminated, status 1 2006-02-01 16:51:34 Task bbstatus terminated, status 1 2006-02-01 16:51:34 Loading hostnames 2006-02-01 16:51:34 Loading saved state 2006-02-01 16:51:34 Setting up network listener on 0.0.0.0:1984 2006-02-01 16:51:34 Setting up signal handlers 2006-02-01 16:51:34 Setting up hobbitd channels 2006-02-01 16:51:34 Setting up logfiles 2006-02-01 16:51:34 Task hobbitclient terminated by signal 15 2006-02-01 16:51:34 Task bbnet terminated by signal 15 2006-02-01 16:52:32 Task hobbitd terminated by signal 6 2006-02-01 16:52:32 Task bbretest terminated by signal 15 2006-02-01 16:52:32 Task clientdata terminated, status 1 2006-02-01 16:52:32 Task rrddata terminated, status 1 2006-02-01 16:52:32 Task rrdstatus terminated, status 1 2006-02-01 16:52:32 Task bbhistory terminated, status 1 2006-02-01 16:52:32 Task bbstatus terminated, status 1 2006-02-01 16:52:32 Loading hostnames 2006-02-01 16:52:32 Loading saved state 2006-02-01 16:52:32 Setting up network listener on 0.0.0.0:1984 2006-02-01 16:52:32 Setting up signal handlers 2006-02-01 16:52:32 Setting up hobbitd channels 2006-02-01 16:52:32 Setting up logfiles 2006-02-01 16:52:37 Task bbdisplay terminated by signal 15 2006-02-01 16:53:32 Task hobbitd terminated by signal 6 2006-02-01 16:53:32 Task bbretest terminated by signal 15 2006-02-01 16:53:32 Task clientdata terminated, status 1 2006-02-01 16:53:32 Task rrddata terminated, status 1 2006-02-01 16:53:32 Task rrdstatus terminated, status 1 2006-02-01 16:53:32 Task bbhistory terminated, status 1 2006-02-01 16:53:32 Task bbstatus terminated, status 1 2006-02-01 16:53:32 Task bbdisplay terminated by signal 15 2006-02-01 16:56:29 Task bb-swap terminated, status 5 2006-02-01 16:56:51 Task bb-httpbench terminated, status 5 2006-02-01 17:01:30 Task bb-swap terminated, status 5 2006-02-01 17:01:52 Task bb-httpbench terminated, status 5 2006-02-01 17:02:36 Loading hostnames 2006-02-01 17:02:36 Loading saved state 2006-02-01 17:02:36 Setting up network listener on 0.0.0.0:1984 2006-02-01 17:02:36 Setting up signal handlers 2006-02-01 17:02:36 Setting up hobbitd channels 2006-02-01 17:02:36 Setting up logfiles 2006-02-01 17:03:33 Task hobbitd terminated by signal 6 2006-02-01 17:03:34 Task clientdata terminated, status 1 2006-02-01 17:03:34 Task rrddata terminated, status 1 2006-02-01 17:03:34 Task rrdstatus terminated, status 1 2006-02-01 17:03:34 Task bbhistory terminated, status 1 2006-02-01 17:03:34 Task bbstatus terminated, status 1 2006-02-01 17:03:34 Loading hostnames 2006-02-01 17:03:34 Loading saved state 2006-02-01 17:03:34 Setting up network listener on 0.0.0.0:1984 2006-02-01 17:03:34 Setting up signal handlers 2006-02-01 17:03:34 Setting up hobbitd channels 2006-02-01 17:03:34 Setting up logfiles 2006-02-01 17:03:34 Task bbdisplay terminated by signal 15 Thanks in advance for your help. Chris

2 1

[hobbit] Users / Procs Graphing problem
by Mike.Rowell＠viatel.com 01 Feb '06

01 Feb '06

Something even more curious, I've just had a closer look at the rrdtool dump and it seems the 5minute data is fully intact, however when it rolls it over to the half hour and increasing time periods that the errors creep in. This looks like it could be an rrd issue, I'll try upgrading rrd first methinks. Regards, Mike Rowell -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: 01 February 2006 13:47 To: hobbit at hswn.dk Subject: Re: [hobbit] Users / Procs Graphing problem Hi Mike, On Wed, Feb 01, 2006 at 01:12:32PM -0000, Rowell, Mike wrote: > I'll pull the data at the client side and see if I can tell whats > going wrong. I was a bit silly when I suggested that you pull the data from the web. An easier way (and it gives me some more interesting data to work with) would be to get it directly from the hobbit daemon. So just run bb 127.0.0.1 "hobbitdlog bad,client,name.cpu" and save this to a file. The reason this is more interesting is because it provides data about when the last status message was received and such. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk For more information about the Viatel Group, please visit www.viatel.com THIS MESSAGE IS INTENDED ONLY FOR THE USE OF THE INTENDED RECIPIENT TO WHICH IT IS ADDRESSED AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, CONFIDENTIAL AND EXEMPT FROM DISCLOSURE. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering the message to the intended recipient, you are notified that any dissemination, distribution or copying of this e-mail is prohibited, and you should delete this e-mail from your system. This message has been scanned for viruses and spam by Viatel MailControl - www.viatel.com

1 0

[hobbit] Users / Procs Graphing problem
by Mike.Rowell＠viatel.com 01 Feb '06

01 Feb '06

Henrik, Thanks for this, the data is actually coming via my own modified client, the data is correct before anyone asks. What I've done is taken the Nagios client as we have extensive nagios monitoring, added a couple of new checks to it and written an ext script that connects to the specified servers, runs the new checks and then sticks the data into Hobbit. We have multiple environments some firewalls some not etc etc so it's the best allround solution I could come up with. I'll pull the data at the client side and see if I can tell whats going wrong. Regards Mike Rowell -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: 01 February 2006 13:02 To: hobbit at hswn.dk Subject: Re: [hobbit] Users / Procs Graphing problem On Wed, Feb 01, 2006 at 12:27:48PM -0000, Rowell, Mike wrote: > I'm building some monitoring solutions at a place I'm working and > during this have noticed an issue with one of the rrd graphs that I > can't figure out.. Everyday at a specific time (different for each > system) there is a 10minute gap in the graph, I've done an rrd dump on > the data and it's appearing as a NaN entry. This means that no data was being fed into the RRD file for 10-15 minutes. > This is only effecting the users / procs graph, anyone got any ideas? Could it be that you are rebooting these servers once a day ? (I know, Unix folks rarely do that - but just in case). Is this with the Hobbit client, or the BB client reporting data ? Since it always happens on the same time for a given server, it would be interesting to see what messages are fed into Hobbit around that time. If you are running the Hobbit client, could you setup a cron job to fetch the client data around that time ? It should run something like wget http://hobbitserver/cgi-bin/bb-hostsvc.sh?CLIENT=bad.client.name and store the output in a file where you can look at it later. Best thing would be if you could run this every minute for 15 minutes around the time this problem occurs. If you are running the BB client, the interesting part is the "cpu" column data that is sent around that time. So something similar, except that the URL you should fetch is http://hobbitserver/cgi-bin/bb-hostsvc.sh?HOSTSVC=bad,client,name.cpu Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk For more information about the Viatel Group, please visit www.viatel.com THIS MESSAGE IS INTENDED ONLY FOR THE USE OF THE INTENDED RECIPIENT TO WHICH IT IS ADDRESSED AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, CONFIDENTIAL AND EXEMPT FROM DISCLOSURE. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering the message to the intended recipient, you are notified that any dissemination, distribution or copying of this e-mail is prohibited, and you should delete this e-mail from your system. This message has been scanned for viruses and spam by Viatel MailControl - www.viatel.com

2 1

Users / Procs Graphing problem
by Mike.Rowell＠viatel.com 01 Feb '06

01 Feb '06

All, Long term bigbrother user, new hobbit convert. I'm building some monitoring solutions at a place I'm working and during this have noticed an issue with one of the rrd graphs that I can't figure out.. Everyday at a specific time (different for each system) there is a 10minute gap in the graph, I've done an rrd dump on the data and it's appearing as a NaN entry. This is only effecting the users / procs graph, anyone got any ideas? If it was effecting all the systems at the same time I might have an idea but it just seems to effect all the systems at different times. Regards, Mike Rowell For more information about the Viatel Group, please visit www.viatel.com THIS MESSAGE IS INTENDED ONLY FOR THE USE OF THE INTENDED RECIPIENT TO WHICH IT IS ADDRESSED AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, CONFIDENTIAL AND EXEMPT FROM DISCLOSURE. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering the message to the intended recipient, you are notified that any dissemination, distribution or copying of this e-mail is prohibited, and you should delete this e-mail from your system. This message has been scanned for viruses and spam by Viatel MailControl - www.viatel.com

2 1

[hobbit] changing behavior of CPU-Test and expand bookkeeping
by brandste＠hebis.de 01 Feb '06

01 Feb '06

Hi, has anybody already tried this? The load-graph in the CPU-test is ok, but we would love to have the utlilization graph instead. This one is a better trigger vor us. What is needed to change the behavior of the client? In that case is recompilation required? Another point concerning the utilization graph, is an expanded bookkeeping. From "configure" my fading memory told me that i could selct between 48, week, month and year.. shame i selcet only the 48 hours.. Is this changeable without a new configure? And where? I would assume the hobbit-server? Thanks in advance Mathias

1 0