- Xymon - Xymon mailman web

Log sizes
by Kevin.Hanrahan＠novainfo.com 06 Mar '05

06 Mar '05

Henrik, I was looking at the size of the logs and some of them are quite large. The largest is the larrd-status.log which has lots of entries like this: 2005-03-05 17:53:37 RRD error updating /home/hobbit/data/rrd/satlcwr01/disk,E.rrd: illegal attempt to update using time 1110063217 when last update time is 1110063217 (minimum one second step) s 1110063217 (minimum one second step) 2005-03-05 17:53:37 RRD error updating /home/hobbit/data/rrd/satlcwr01/users.rrd: illegal attempt to update using time 1110063217 when last update time is 1110063217 (minimum one second step) 2005-03-05 17:53:37 RRD error updating /home/hobbit/data/rrd/satlcwr01/disk,C.rrd: illegal attempt to update using time 1110063217 when last update time is 1110063217 (minimum one second step) 2005-03-05 17:53:37 RRD error updating /home/hobbit/data/rrd/satlcwr01/disk,E.rrd: illegal attempt to update using time 1110063217 when last update time is 1110063217 (minimum one second step) 1) what does this mean? 2) Other than troubleshooting, are the logs used for any other function? Can I delete them? Thanks Kevin Note: The information contained in this email and in any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. The recipient should check this email and any attachments for the presence of viruses. Sender accepts no liability for any damages caused by any virus transmitted by this email. If you have received this email in error, please notify us immediately by replying to the message and delete the email from your computer. This e-mail is and any response to it will be unencrypted and, therefore, potentially unsecure. Thank you. NOVA Information Systems, Inc.

2 1

new version of hobbit client
by brianlynch＠gmail.com 06 Mar '05

06 Mar '05

Henrik, This is pretty minor, but when do you expect to rev the hobbit client? The current version seems a bit shaky on the compile. Are you planning on changing the paradigm used by the bb clients (i.e. collection of scripts calling 'bb')? Best, Brian

3 2

completely confused
by trotunda＠opushealthcare.com 05 Mar '05

05 Mar '05

Ok, maybe I have spent too much time looking at too many different man pages for too many different products. Whatever the case may be, I could really use a cold slap in the face. I want to monitor anything and everything I can on 100+ servers (numbers growing monthly) all over the US. Some are HP-UX, some are Linux, some are Windows. I am focused on HP-UX and Linux for now. I would like to use bb-central so that I do not have to deploy clients across my enterprise, but I won't be so stubborn that I won't use bb-clients, as I really need to have a hand full of remote servers monitored (cpu, disk space, etc) by next Thursday. I am trying to fend off a "wannabe" administrator who is pushing Nagios, which I do not want to deploy. Ok, here are the questions (please forgive my ignorance)..... - It looks like I need to install Hobbit and BB-Client to monitor the OS level on the Hobbit server. True? - Can I "hobbit" enable my applications to report status from remote servers without any middleware (bb-client)? o I would think this could be a "one liner" in my apps (20-30 of them) to report their internal status.???? - Where can I see the message format for sending my own messages to the display server, if that is possible? - It looks like if I want to use bb-central: o Do I need both bb-server and bb client on the display server? o Should I modify bb-central to not use BB or maybe just rewrite that functionality to fit hobbit? Any advice, comments, suggestion welcome. Thanks, Tim Rotunda I.T. & Systems Integration Manager Opus Healthcare Solutions, Inc. Austin, Texas 78759

2 1

Purple after RC5 install
by cmorrison＠corniceco.com 05 Mar '05

05 Mar '05

I installed RC5 on a new server today after running RC3 for several weeks without a problem. Initially RC5 picked up right away and was fine. After about an hour I now get purple on all conn, ssh, http and bdb items for every server monitored (including the hobbit server) on the main page. Each sub page I go to shows a purple background, but OK on each test. This is a new install, I wiped the RC3 server, installed an OS upgrade (formatted everything) and reinstalled RC5 as new, using only the old bb-hosts file. Any hints on what this could be ? Chuck Morrison

2 3

[hobbit] BOARDBUSY messages in logs
by dstuffle＠deltafaucet.com 04 Mar '05

04 Mar '05

I see, thanks. My main goal is trying to get the mail acks to work. That's why I was digging through the log files. Anyway, my mail acks are only sometimes working. I haven't been able to find a pattern to it yet so I will need to investigate more. I do know though that adding MESSAGE=something to the subject line when replying to an email alert doesn't work. Should that work, or am I misunderstanding something? I get this in the acklog (all one line): 1109963670 126510 30 126510 np_filename_not_used zenmaster.memory red ~~~~~~~~~~~~~~\nAcked by: "Stuffle, David" dstuffle at deltafaucet.com Several times that I acked, I get nothing at all in the acklog. Thanks, David -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Friday, March 04, 2005 12:13 PM To: hobbit at hswn.dk Subject: Re: [hobbit] BOARDBUSY messages in logs On Fri, Mar 04, 2005 at 12:00:22PM -0500, Stuffle, David wrote: > I am still seeing BOARDBUSY messages in my logs like the following: > larrd-status.log: > 2005-03-04 11:36:11 Tried to down BOARDBUSY: Invalid argument This one is harmless, and happens when Hobbit shuts down. The master daemon (hobbitd) sends a shutdown-message to all of the hobbitd_channel worker modules, then marks the shared memory segments used for communication with the modules for deletion. Depending on how fast the workers respond, they may try to access the shared memory segment when it has been deleted, and you'll get this message. > My log files don't seem to be getting updated properly and I'm > wondering if this has anything to do with it. I'm on Fedora. Which log files, specifically ? The only files that Hobbit normally updates while running are the historical logs in the data/hist/ and data/histlogs/ directories - and they only get updated when there are status changes. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Finally, the recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

3 2

[hobbit] bb-central
by trotunda＠opushealthcare.com 04 Mar '05

04 Mar '05

Dan, Thanks for the info. I just started my learning curve a few days ago and already have 32 servers monitored. I am under the gun to prove Hobbit a better tool than Nagios and it looks to me like incorporating bb-central is all the ammo I will need. Very powerful if I can get it to work well. Thanks again, Tim Austin, Texas. -----Original Message----- From: Daniel J McDonald [mailto:dan.mcdonald at austinenergy.com] Sent: Friday, March 04, 2005 12:33 PM To: Hobbit List Subject: Re: [hobbit] bb-central On Fri, 2005-03-04 at 11:49 -0600, Tim Rotunda wrote: > Can anyone tell me if bb-central will work with Hobbit if I do not > have bbclient loaded? It looks like it but install says to change > bbrun.sh script and I don’t have one of those. In hobbit, it would be hobbitlaunch.cfg that you would fiddle with. You just create a new entry like this one I made for bbmrtg.pl: [bbmrtg] ENVFILE /var/hobbit/server/etc/hobbitserver.cfg NEEDS hobbitd CMD $BBHOME/ext/bbmrtg.pl LOGFILE $BBSERVERLOGS/bbmrtg.log INTERVAL 5m > Do I just need to see what the format of that is? Am I barking up > the wrong tree here? I have 100 plus servers I need to monitor and I > really don’t want to deploy anything across all of them. Ditto. I haven't played with bb-central under hobbit yet, but that's coming up soon on my agenda. Any pointers you come up with are most appreciated. -- Daniel J McDonald, CCIE # 2495, CNX Austin Energy dan.mcdonald at austinenergy.com To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

2 1

recovery emails on alerts which don't generate pages == undesired behaviour
by blysik＠shutterfly.com 04 Mar '05

04 Mar '05

Hi, Previously, we've only wanted to alert on red and purples, and then send recovery emails when it changes out of a red or purple state. This was easily accomplished by setting --alertcolors to red,purple. Recently however, I've gotten some requests from people who want to get alerted on a few monitors when they yellow. No problem, I thought. I'll just change --alertcolors back to default, and then add COLOR=red,purple to all the existing alert definitions to start. This caused the problem where a monitor would go into a yellow state (not causing a page because COLOR=red,purple) and then go back to green (which would then send a recovery page). This current behaviour doesn't make sense. Why would I want to be alerted on a recovery of something that never generated an alert in the first place? It would make more sense if the recovery email condition was tied to the COLOR definition. So an alert with COLOR=red,yellow,purple (the default) would send a recovery message on leaving the red, yellow, or purple state to green, blue, or clear. And an alert with COLOR=red,purple would only send a recovery message if it was leaving a red or purple state. Or maybe I'm braindead and can't see how this can be accomplished currently. Comments? -- Bruce Z. Lysik <blysik at shutterfly.com> Operations Engineer

3 3

[hobbit] Iostat graphs
by Kevin.Hanrahan＠novainfo.com 04 Mar '05

04 Mar '05

You might be right...I honestly don't remember which one is on that particular server. I have had problems getting a single I/O script to work consistently across all our *NIX platforms (solaris, linux, and aix....maybe SCO someday) so I am really just looking for a good I/O tool that works cross-platform...and the graphs would be a major bonus too. Suggestions? Kevin -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Friday, March 04, 2005 1:32 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Iostat graphs On Fri, Mar 04, 2005 at 12:24:37PM -0500, Kevin.Hanrahan at novainfo.com wrote: > Here is the output of one of the systems: This doesn't look like the iostat output from the LARRD bf-iostat.sh script. More like something from the vmio script, but not quite. So it's quite understandable that you don't get any graphs, because this is an input format that is simply not recognized. If you can point me to the script you're using and some of the data you want to graph, I'll see if I can come up with something. But much of the data should already be available if you have the vmstat graphs (particularly the vmstat "wa" column for how much cpu time is spent in i/o wait). Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk Note: The information contained in this email and in any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. The recipient should check this email and any attachments for the presence of viruses. Sender accepts no liability for any damages caused by any virus transmitted by this email. If you have received this email in error, please notify us immediately by replying to the message and delete the email from your computer. This e-mail is and any response to it will be unencrypted and, therefore, potentially unsecure. Thank you. NOVA Information Systems, Inc.

1 0

bb-central
by trotunda＠opushealthcare.com 04 Mar '05

04 Mar '05

Can anyone tell me if bb-central will work with Hobbit if I do not have bbclient loaded? It looks like it but install says to change bbrun.sh script and I don't have one of those. Do I just need to see what the format of that is? Am I barking up the wrong tree here? I have 100 plus servers I need to monitor and I really don't want to deploy anything across all of them. Thanks for any info, Tim

2 1

[hobbit] Iostat graphs
by Kevin.Hanrahan＠novainfo.com 04 Mar '05

04 Mar '05

Here is the output of one of the systems: dwprod|iostat|green||1109955157|1109956353|1109958153|0|0|10.164.134.32|-1|| green Fri Mar 4 12:19:43 EST 2005 Virtual Memory Info (/usr/bin/vmstat 3 10) System Configuration: lcpu=4 mem=12288MB kthr memory page faults cpu ----- ----------- ------------------------ ------------ ----------- r b avm fre re pi po fr sr cy in sy cs us sy id wa 1 2 1742256 221056 0 0 0 57 187 0 1118 11551 2259 13 5 65 17 2 4 1743281 220039 0 0 0 3638 35155 0 1383 21640 1561 28 14 2 56 3 4 1743464 219850 0 0 0 5302 59687 0 1380 32418 2129 30 22 11 36 3 5 1744448 218898 0 0 0 8531 68216 0 2418 9874 2417 14 15 12 59 2 4 1746221 217134 0 0 0 5888 53164 0 1180 56781 1929 28 14 19 39 2 3 1746231 217070 0 0 0 4739 29844 0 894 18563 2034 35 15 14 35 3 2 1746961 216412 0 0 0 6969 42484 0 986 14507 2087 51 26 7 17 2 2 1744307 219070 0 0 0 6787 47050 0 913 16042 1918 49 16 12 24 2 2 1744324 219032 0 0 0 5609 35614 0 797 15220 1527 34 12 24 30 4 2 1744324 219044 0 0 0 5046 28956 0 777 13252 1590 32 11 22 34 &green There's a avg of 2 processes in run queue. &green There's a avg of 3 blocked processes. &green There's a avg swap of 0 available. &green There's a avg of 0 k paged in. &green There's a avg of 0 k paged out. &green There's a avg scan rate of 44463 present. &green There's a avg sys cpu time of 0 percent. Disk I/O Statistics (/usr/bin/iostat -t 2 5) System configuration: lcpu=4 tty: tin tout avg-cpu: % user % sys % idle % iowait 0.3 25.6 12.7 5.5 64.8 17.0 0.0 0.0 32.2 12.1 21.2 34.4 0.0 0.0 22.6 17.2 6.6 53.6 0.0 0.0 34.8 13.8 3.8 47.8 0.0 0.0 37.8 18.8 4.5 38.9 &green There's an avg I/O wait time of 43. I do get an IOSTAT column with this data in it but the graph at the bottom has always been blank. -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Friday, March 04, 2005 11:49 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Iostat graphs On Fri, Mar 04, 2005 at 11:39:30AM -0500, Kevin Hanrahan wrote: > Has anyone ever gotten iostat to put any data into the larrd graphs? > I get plenty of data for iostat but the graphs are always blank. > Anybody got any advice? Lets see what data you're sending into the iostat handler. Could you do a ~hobbit/server/bin/bb 127.0.0.1 "hobbitdlog HOSTNAME.iostat" and show us the outpu ? If the iostat data is sent as "data" messages (i.e. you dont have an iostat column), then you'll need to snoop the messages going out on the hobbit data-channel: Login as the hobbit user, then run server/bin/bbcmd sh hobbitd_channel --channel=data cat |tee log |grep iostat Wait for a while until an iostat message shows up, then pick it out of the logfile. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk Note: The information contained in this email and in any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. The recipient should check this email and any attachments for the presence of viruses. Sender accepts no liability for any damages caused by any virus transmitted by this email. If you have received this email in error, please notify us immediately by replying to the message and delete the email from your computer. This e-mail is and any response to it will be unencrypted and, therefore, potentially unsecure. Thank you. NOVA Information Systems, Inc.

2 1