- Xymon - Xymon mailman web

[hobbit] Status Unavailable - again
by ReifJefferyM＠JohnDeere.com 11 Jul '05

11 Jul '05

Stefan, I don't know if this is your problem. I just suggested it as a possibility for you to consider. I originally saw random lockups on a BB system after upgrading from Redhat 7 to (8 or 9, I don't remember). I thought this might be possible on other versions or distributions as well. The solution in my case was to either: 1) Down-grade the compiler to match the OS . 2) Recompile the kernel. 3) Change OS versions (sounds like you may have tried this). I hope this helps in some way. Good luck on your problem. Jeff -----Original Message----- From: Stefan Loos [mailto:stefan_loos at hotmail.com] Sent: Monday, July 11, 2005 8:24 AM To: hobbit at hswn.dk Subject: RE: [hobbit] Status Unavailable - again Hello Jeffery, do you think we see this issue because the kernel was built with a different compiler than I was using for building the hobbit server? @Henrik - do you think this could be the problem? Regards, Stefan >From: "Reif Jeffery M" <ReifJefferyM at JohnDeere.com> >Reply-To: hobbit at hswn.dk >To: hobbit at hswn.dk >Subject: RE: [hobbit] Status Unavailable - again >Date: Mon, 11 Jul 2005 07:20:12 -0500 > >This is a shot in the dark - I had a lockup problem on a BB system where >the kernel was compiled with a different version compiler than was >included with the system and there were some IPC-related changes in >compiler versions. > >-----Original Message----- >From: Stefan Loos [mailto:stefan_loos at hotmail.com] >Sent: Monday, July 11, 2005 7:12 AM >To: hobbit at hswn.dk >Subject: Re: [hobbit] Status Unavailable - again > >Hi Henrik, > >we have several own-written scripts (mostly in perl) which monitor >oracle >instances, bea weblogic servers. There is one for hardware monitoring - >HP >(Intel) and Sun Servers (prtdiag based) and some are just the output of >a >http request to the software running on that weblogic servers. >I have just one server for testing, it's a HP DL 360. We are running >Redhat >Enterprise Server 3 but I've tried it with a SuSE 9.3 too. > >Regards, > >Stefan > > &gt;From: henrik at hswn.dk (Henrik Stoerner) &gt;Reply-To: >hobbit at hswn.dk &amp ;gt;To: hobbit at hswn.dk &gt;Subject: Re: [hobbit] >Status >Unavailable - again &gt;Date: Mon, 11 Jul 2005 13:08:59 >+0200 &gt; &gt;Hi Stefan, &gt; &gt;On Mon, Jul 11, 2005 at >09:36:42AM +0000, Stefan Loos wrote: &gt; &gt; It would be great if >you >can put me in cc. If you want I can try to assist &gt; &gt; you. I'm >at a >point where I don't know what to try anymore. I think it &gt; &gt; >isn't >easy for Henrik to find this issue - I have no coredumps and >nothing &gt; >&gt; in the logfile what could help. &gt; &gt;Yes, this is a >really >nasty problem. Vernon and I though we had it &gt;nailed down by the >end >of last week, but there's more to it than &gt;what we found >then. &gt; &gt;What kind of external scripts run on your clients, >apart from the BB &gt;client ? The current suspicion is that this is >triggered by a status &gt;message that is handled badly by Hobbit >causing >this lock-up. So I'm &gt;trying to see if there might be something in > >common between your setups. &gt; &gt;And what kind of system are >you >running Hobbit on ? If Linux, which &gt;distribution ? Another >suspicion >I have is that this might be a &gt;problem with the implementation of > >SysV IPC >semaphores. &gt; &gt; &amp ;gt;Regards, &gt;Henrik &gt; &gt >; &gt;To >unsubscribe from the hobbit list, send an e-mail >to &gt;hobbit-unsubscribe at hswn.dk &a mp;gt; &gt; > > > >To unsubscribe from the hobbit list, send an e-mail to >hobbit-unsubscribe at hswn.dk > > > > > >To unsubscribe from the hobbit list, send an e-mail to >hobbit-unsubscribe at hswn.dk > > To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

[hobbit] Status Unavailable - again
by v.everett＠afgonline.com.au 11 Jul '05

11 Jul '05

English has no shame. It will borrow beg or steal words from any language :-) Mensch, was taken from Yiddish - A hybrid of old German and Hebrew.

1 0

[hobbit] Status Unavailable - again
by v.everett＠afgonline.com.au 11 Jul '05

11 Jul '05

Yep. I did. Between myself and Henrik, we have tried a number of versions, and a few special diagnostic versions of Hobbit. We have been working mostly off-list because we have been exchanging potentially confidential information, and don't believe that our failure to diagnose a problem is of general interest. (I am sure a full account of this sad tale will be posted once it is resolved.) Kudos to Henrik though. I think he has really tried. He has worked tirelessly to try and resolve this issue, which I believe is very admirable, when you consider his reward for all this hard work. Henrik is a true mensch. (Yes, that's an English word. Look it up) So far, we have not been able to identify the root cause of the problem. Henrik was going to have a look at some of the messages that came from one of the hosts, and get back to me. Stefan, if you are interested in assisting us with this, then myself and Henrik can cc you in our off-list exchanges. Have you got any theories as to the cause? Regards Vernon -----Original Message----- From: Stefan Loos [mailto:stefan_loos at hotmail.com] Sent: Monday, 11 July 2005 4:21 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Status Unavailable - again did you use the hobbitd from the snapshot? _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ NOTICE: This message and any attachments are confidential and may contain copyright material of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the addressee and any other named recipient. If you are not the intended recipient, any use, distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached to this message is not waived or lost by reason of the mistaken transmission or delivery to any unintended party. If you have received this message in error, please notify the author immediately or contact Australian Finance Group on +61 8 9420 7888.

3 3

[hobbit] Status Unavailable - again
by v.everett＠afgonline.com.au 11 Jul '05

11 Jul '05

Try http://www.onelook.com/?w=mensch&ls=a :-) Once Hobbit hangs, it refuses to core. The only thing that will clobber it, is a kill -9 When it hangs, it hangs good. The logs are devoid of useful info too. Even with the latest build. This is probably the most frustrating issue I have ever seen. Regards Vernon -----Original Message----- From: Stefan Loos [mailto:stefan_loos at hotmail.com] Sent: Monday, 11 July 2005 5:37 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Status Unavailable - again It would be great if you can put me in cc. If you want I can try to assist you. I'm at a point where I don't know what to try anymore. I think it isn't easy for Henrik to find this issue - I have no coredumps and nothing in the logfile what could help. And I never had any doubt that Henrik does a great job! (I think my English is not good enough to say it in other words) So if there is anything what I can do to solve this problem.... I've tried to lookup "mensch" but I think I'm using the wrong sites - german english dictionaries always recognize mensch as a german word ;-) Regards, Stefan >From: "Vernon Everett" <v.everett at afgonline.com.au> >Reply-To: hobbit at hswn.dk >To: <hobbit at hswn.dk> >Subject: RE: [hobbit] Status Unavailable - again >Date: Mon, 11 Jul 2005 16:37:28 +0800 > >Yep. >I did. >Between myself and Henrik, we have tried a number of versions, and a few >special diagnostic versions of Hobbit. >We have been working mostly off-list because we have been exchanging >potentially confidential information, and don't believe that our failure >to diagnose a problem is of general interest. (I am sure a full account >of this sad tale will be posted once it is resolved.) > >Kudos to Henrik though. >I think he has really tried. He has worked tirelessly to try and resolve >this issue, which I believe is very admirable, when you consider his >reward for all this hard work. >Henrik is a true mensch. (Yes, that's an English word. Look it up) > >So far, we have not been able to identify the root cause of the problem. >Henrik was going to have a look at some of the messages that came from >one of the hosts, and get back to me. > >Stefan, if you are interested in assisting us with this, then myself and >Henrik can cc you in our off-list exchanges. > >Have you got any theories as to the cause? > >Regards > Vernon > > > >-----Original Message----- >From: Stefan Loos [mailto:stefan_loos at hotmail.com] >Sent: Monday, 11 July 2005 4:21 PM >To: hobbit at hswn.dk >Subject: RE: [hobbit] Status Unavailable - again > >did you use the hobbitd from the snapshot? > > >_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > >NOTICE: This message and any attachments are confidential and may contain copyright material >of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the >addressee and any other named recipient. If you are not the intended recipient, any use, >distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached >to this message is not waived or lost by reason of the mistaken transmission or delivery to any >unintended party. If you have received this message in error, please notify the author immediately or >contact Australian Finance Group on +61 8 9420 7888. > > >To unsubscribe from the hobbit list, send an e-mail to >hobbit-unsubscribe at hswn.dk > > To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ NOTICE: This message and any attachments are confidential and may contain copyright material of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the addressee and any other named recipient. If you are not the intended recipient, any use, distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached to this message is not waived or lost by reason of the mistaken transmission or delivery to any unintended party. If you have received this message in error, please notify the author immediately or contact Australian Finance Group on +61 8 9420 7888.

2 1

purple/clear problems?!
by peter.welter＠gmail.com 11 Jul '05

11 Jul '05

Hi all, Hope you cold shine a light on this one. I'm running Hobbit 4.0.4 on Solaris 8. I've enabled reporting as follows with macro's in hobbit-alert.cfg: $UNIXDAY=MAIL someone at somewhere.nl TIME=*:0000:2359 REPEAT=1d RECOVERED $UNIXNIGHT=MAIL someone at somewhere.nl TIME=*:0000:2359 DURATION>30m REPEAT=60m RECOVERED SERVICE=!cpu,!msgs COLOR=!yellow This morning I found in my mailbox about 4000 mails reporting stopped & recovered. What happened was that the connection for 'host' failed. I've got 1 email of that event as it should. But mainly, until the connection re-established last night, a bouncing effect for other than the conn-test. I could build a dependancy (don't warn if no-conn) but that is rather annoying since I have many hosts to manage. I just don't fully understand why the other tests are bouncing whereas the conn-test turns off on friday and on last night. See below: ... Bigbrother account Hobbit [705353] host:disk stopped reporting (PURPLE) za 25-06-05 6:04 1 KB Bigbrother account Hobbit [735704] host:msgs stopped reporting (PURPLE) za 25-06-05 6:04 1 KB Bigbrother account Hobbit [115710] host:procs stopped reporting (PURPLE) za 25-06-05 6:04 1 KB Bigbrother account Hobbit [963024] host:cpu stopped reporting (PURPLE) za 25-06-05 6:04 1 KB Bigbrother account Hobbit host:msgs recovered za 25-06-05 6:05 1 KB Bigbrother account Hobbit host:procs recovered za 25-06-05 6:05 1 KB Bigbrother account Hobbit host:disk recovered za 25-06-05 6:05 1 KB Bigbrother account Hobbit host:cpu recovered za 25-06-05 6:05 1 KB Bigbrother account Hobbit [982927] host:procs stopped reporting (PURPLE) za 25-06-05 6:09 1 KB Bigbrother account Hobbit [276744] host:disk stopped reporting (PURPLE) za 25-06-05 6:09 1 KB Bigbrother account Hobbit [850873] host:msgs stopped reporting (PURPLE) za 25-06-05 6:09 1 KB Bigbrother account Hobbit [612113] host:cpu stopped reporting (PURPLE) za 25-06-05 6:09 1 KB The process check for example turn purple, of course, but then a clear status appears: ... Sun Jun 26 20:09:44 2005 clear 0:04:43 Sun Jun 26 20:09:27 2005 purple 0:00:17 Sun Jun 26 20:04:32 2005 clear 0:04:55 Sun Jun 26 20:04:27 2005 purple 0:00:05 Sun Jun 26 20:00:22 2005 clear 0:04:05 Sun Jun 26 19:59:26 2005 purple 0:00:56 ... Thanks, Peter

2 4

[hobbit] Status Unavailable - again
by v.everett＠afgonline.com.au 11 Jul '05

11 Jul '05

I am still here, and still having major problems :-( I thought we had isolated the problem to messages from a particular monitored client, but after a few hours of smooth running, with messages from that client disabled, it failed again. Regards Vernon -----Original Message----- From: Stefan Loos [mailto:stefan_loos at hotmail.com] Sent: Monday, 11 July 2005 3:49 PM To: hobbit at hswn.dk Subject: [hobbit] Status Unavailable - again Hello Henrik, after the server runs for a few days the error came back (even with the hobbitd of the snapshot-version you gave me). It would be interesting to hear from Vernon if his server still has this problem.... Regards, Stefan To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ NOTICE: This message and any attachments are confidential and may contain copyright material of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the addressee and any other named recipient. If you are not the intended recipient, any use, distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached to this message is not waived or lost by reason of the mistaken transmission or delivery to any unintended party. If you have received this message in error, please notify the author immediately or contact Australian Finance Group on +61 8 9420 7888.

2 2

[hobbit] Status Unavailable
by v.everett＠afgonline.com.au 11 Jul '05

11 Jul '05

Never actually checked that. Nope, hobbitd refuses to die. You got any ideas? -----Original Message----- From: Stefan Loos [mailto:stefan_loos at hotmail.com] Sent: Monday, 4 July 2005 2:43 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Status Unavailable And can you stop the hobbit server with hobbit.sh or is one process still running after that? _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ NOTICE: This message and any attachments are confidential and may contain copyright material of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the addressee and any other named recipient. If you are not the intended recipient, any use, distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached to this message is not waived or lost by reason of the mistaken transmission or delivery to any unintended party. If you have received this message in error, please notify the author immediately or contact Australian Finance Group on +61 8 9420 7888.

3 7

logfiles
by tlp-hobbit＠holme-pedersen.dk 08 Jul '05

08 Jul '05

With hobbit you have the option not to create the bbvar hist log files and in the hobbitlaunch.cfg file you have a task to create this if you need it. My question is how can I have hobbit only to some logfiles and not all ? is it possible ie adding a LOG entry to the host in bb-hosts Regards, Thomas

2 1

[hobbit] RE : [hobbit] RE : [hobbit] I can't get any graph on hobbit
by nfigaro＠effigie.fr 08 Jul '05

08 Jul '05

I just commented out the "COMMENT" part in hobbitd/hobbitgraph.cgi, and it works. (I can see my graphs on hobbit on AIX). Here are the lines commented : +815 /* #ifdef RRDTOOL12 */ +816 /* strftime(timestamp, sizeof(timestamp), "COMMENT:Updated\\: %d-%b-%Y %H\\:%M\\:%S", localtime(&now)); */ +817 /*#else */ +818 /* strftime(timestamp, sizeof(timestamp), "COMMENT:Updated: %d-%b-%Y %H:%M:%S", localtime(&now)); */ +819 /*#endif */ +820 /* rrdargs[argi++] = strdup(timestamp); */ But I don't know why the COMMENT generates an error in rrd on AIX. Thanks Henrik and Rolf for the replies. NF -----Message d'origine----- De : FIGARO Nicolas [mailto:nfigaro at effigie.fr] Envoyé : vendredi 8 juillet 2005 14:35 À : hobbit at hswn.dk Objet : [hobbit] RE : [hobbit] RE : [hobbit] I can't get any graph on hobbit I started with hobbit 4.0.4, and I use the rrdtool 1.2.10. I can create an rrd graph using this command line: rrdtool graph ~hobbit/server/www/test.png DEF:avg=la.rrd:la:AVERAGE CDEF:la=avg,100,/ AREA:la#00CC00:"CPU Load Average" -u 1.0 I don't know if I can comment some parts of the hobbitgraph.cfg file, or increase a debug level somewhere . (but the garbage message comes from the rrd_graph_helper.c file of rrdtool, perhaps I should send the problem to the rrd mailing list. ). NF -----Message d'origine----- De : Henrik Stoerner [mailto:henrik at hswn.dk] Envoyé : vendredi 8 juillet 2005 13:56 À : hobbit at hswn.dk Objet : Re: [hobbit] RE : [hobbit] I can't get any graph on hobbit On Fri, Jul 08, 2005 at 01:48:44PM +0200, FIGARO Nicolas wrote: > <html><head><title>Invalid request</title></head> <body>Garbage ': > 08-Jul-2005 13:38:46' after command: > COMMENT:Updated: 08-Jul-2005 13:38:46</body></html> The latest (4.0.4) Hobbit version has some changes to hobbitgraph.cfg which were needed for RRDtool 1.2.x. If you originally installed an older version (4.0, 4.0.2, maybe even 4.0.3) then it will not update your hobbitgraph.cfg because it's a file that you might customize yourself. So the easiest solution - if you haven't done any modifications - is to just copy the hobbit-4.0.4/hobbitd/etcfiles/hobbitgraph.cfg to your ~hobbit/etc/directory. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

[hobbit] RE : [hobbit] I can't get any graph on hobbit
by nfigaro＠effigie.fr 08 Jul '05

08 Jul '05

I started with hobbit 4.0.4, and I use the rrdtool 1.2.10. I can create an rrd graph using this command line: rrdtool graph ~hobbit/server/www/test.png DEF:avg=la.rrd:la:AVERAGE CDEF:la=avg,100,/ AREA:la#00CC00:"CPU Load Average" -u 1.0 I don't know if I can comment some parts of the hobbitgraph.cfg file, or increase a debug level somewhere . (but the garbage message comes from the rrd_graph_helper.c file of rrdtool, perhaps I should send the problem to the rrd mailing list. ). NF -----Message d'origine----- De : Henrik Stoerner [mailto:henrik at hswn.dk] Envoyé : vendredi 8 juillet 2005 13:56 À : hobbit at hswn.dk Objet : Re: [hobbit] RE : [hobbit] I can't get any graph on hobbit On Fri, Jul 08, 2005 at 01:48:44PM +0200, FIGARO Nicolas wrote: > <html><head><title>Invalid request</title></head> <body>Garbage ': > 08-Jul-2005 13:38:46' after command: > COMMENT:Updated: 08-Jul-2005 13:38:46</body></html> The latest (4.0.4) Hobbit version has some changes to hobbitgraph.cfg which were needed for RRDtool 1.2.x. If you originally installed an older version (4.0, 4.0.2, maybe even 4.0.3) then it will not update your hobbitgraph.cfg because it's a file that you might customize yourself. So the easiest solution - if you haven't done any modifications - is to just copy the hobbit-4.0.4/hobbitd/etcfiles/hobbitgraph.cfg to your ~hobbit/etc/directory. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0