FW: Re: [hobbit] bbdisplay problems after adding some new hosts
Hello,
is the number of tests per host limited? I've cleaned my bb-hosts and as I add one host with many tests. I disabled all customized tests and tried to find out if its a single test. But all test run by itself didn't crash the hobbit server - running all together did! The host runs about 20 tests.
Regards,
Stefan
<br><br><br>>From: "Stefan Loos"
<stefan_loos at hotmail.com><br>>Reply-To: hobbit at hswn.dk<br>>To:
hobbit at hswn.dk<br>>Subject: Re: [hobbit] bbdisplay problems after adding
some new hosts<br>>Date: Wed, 11 May 2005 09:49:30
+0000<br>><br>>Hi Henrik,<br>><br>>now the errors in the
hobbitlaunch.log are gone but in <br>>bb-display.log are still there. And
another strange thing - since I <br>>reenabled the bbdisplay this morning
I didn't see any host at the <br>>hobbit server! Just the subpages and
groups are there.<br>><br>>Regards,<br>><br>>Stefan
Loos<br>><br>><br><br><br>>From:
henrik at hswn.dk (Henrik <br>>Stoerner)<br>>Reply-To:
hobbit at hswn.dk<br>>To:
<br>>hobbit at hswn.dk<br>>Subject: Re: [hobbit] bbdisplay
problems after <br>>adding some new hosts<br>>Date: Wed, 11
May 2005 11:06:13 <br>>+0200<br>><br>>Could you
try removing the <br>>"HEARTBEAT" line from
hobbitlaunch.cfg and<br>>see if <br>>things run OK after
that
<br>>?<br>><br>><br>>Regards,<br>>Henrik<br>><br>>On
<br>>Wed, May 11, 2005 at 07:41:37AM +0000, Stefan Loos
wrote:<br>> <br>>> Hi,<br>>
><br>> > yesterday I add some new hosts to
<br>>my hobbit-server and short after that<br>> >
hobbit had some <br>>problems.<br>> > Here is what
hobbitlauch.log says:<br>> <br>>><br>>
> 2005-05-11 09:11:18 Heartbeat lost for task <br>>hobbitd,
bouncing it<br>> > 2005-05-11 09:11:18 Task bbretest
<br>>started with PID 4523<br>> > 2005-05-11 09:11:23
Heartbeat <br>>lost for task hobbitd, killing it<br>>
> 2005-05-11 09:11:23 <br>>Task bbdisplay started with PID
4524<br>> > 2005-05-11 <br>>09:11:23 Task hobbitd
terminated by signal 9<br>> > 2005-05-11
<br>>09:11:23 Task hobbitd started with PID 4525<br>>
> 2005-05-11 <br>>09:11:23 Loading hostnames<br>>
> 2005-05-11 09:11:23 Loading <br>>saved state<br>>
> 2005-05-11 09:11:23 Setting up network <br>>listener on
0.0.0.0:1984<br>> > 2005-05-11 09:11:23 Setting up
<br>>signal handlers<br>> > 2005-05-11 09:11:23
Setting up hobbitd <br>>channels<br>> > 2005-05-11
09:11:23 Setting up <br>>logfiles<br>> > 2005-05-11
09:11:28 Task bbhistory started <br>>with PID 4527<br>>
> 2005-05-11 09:11:28 Task bbenadis started <br>>with PID
4528<br>> > 2005-05-11 09:11:28 Task bbpage started
<br>>with PID 4530<br>> > 2005-05-11 09:11:28 Task
larrdstatus <br>>started with PID 4532<br>> >
2005-05-11 09:11:28 Task <br>>larrddata started with PID
4534<br>> > 2005-05-11 09:12:18 <br>>Task bbretest
started with PID 4541<br>> > 2005-05-11 09:12:23
<br>>Task bbdisplay started with PID 4542<br>> >
2005-05-11 <br>>09:12:43 Heartbeat lost for task hobbitd, bouncing
it<br>> > <br>>2005-05-11 09:12:48 Heartbeat lost for
task hobbitd, killing <br>>it<br>> > 2005-05-11
09:12:48 Task hobbitd terminated by <br>>signal 9<br>>
> 2005-05-11 09:12:48 Task bbdisplay terminated <br>>by signal
15<br>> ><br>> > So I tried to find
out which <br>>component causes the problem and
disabled<br>> > everything in <br>>hobbitlauch.cfg
and reenabled one by one.<br>> > I found out
<br>>that everytime I enabled bbdisplay those errors
occour.<br>> > <br>>The bb-display.log looks like
this:<br>> ><br>> >
<br>>2005-05-11 09:09:48 Whoops ! bb failed to send message -
<br>>timeout<br>> > 2005-05-11 09:09:48 hobbitd
status-board not <br>>available<br>> > 2005-05-11
09:09:53 Whoops ! bb failed to <br>>send message -
timeout<br>> > 2005-05-11 09:10:53 Whoops ! bb
<br>>failed to send message - timeout<br>> >
2005-05-11 09:10:53 <br>>hobbitd status-board not
available<br>> > 2005-05-11 09:11:23 <br>>Could not
connect to bbd at 10.207.193.41:1984 -<br>> >
<br>>Connection refused<br>> > 2005-05-11 09:11:23
Whoops ! bb <br>>failed to send message - Connection
failed<br>> > 2005-05-11 <br>>09:11:23 hobbitd
status-board not available<br>> > 2005-05-11
<br>>09:11:23 Could not connect to bbd at 10.207.193.41:1984
-<br>> > <br>>Connection refused<br>>
> 2005-05-11 09:11:23 Whoops ! bb <br>>failed to send message -
Connection failed<br>> ><br>> >
<br>>I also found some core files in ~server/tmp but I'm pretty shure
<br>>they came<br>> > from killing hobbit -
nevertheless I've run <br>>the gdb util:<br>>
><br>> > GNU gdb Red Hat Linux
<br>>(6.1post-1.20040607.52rh)<br>> > Copyright 2004
Free Software <br>>Foundation, Inc.<br>> > GDB is
free software, covered by the <br>>GNU General Public License, and you
are<br>> > welcome to <br>>change it and/or
distribute copies of it under certain<br>> >
<br>>conditions.<br>> > Type "show
copying" to see the <br>>conditions.<br>> >
There is absolutely no warranty for GDB. <br>>Type "show
warranty" for details.<br>> > This GDB
<br>>was configured as "i386-redhat-linux-gnu"...Using
<br>>host<br>> > libthread_db library
<br>>"/lib/tls/libthread_db.so.1".<br>>
><br>> > <br>>Core was generated by `hobbitd
--debug<br>> >
<br>>--pidfile=/var/log/hobbit/hobbitd.pid
<br>>--restart=/usr/local/hobb'.<br>> > Program
terminated with <br>>signal 6, Aborted.<br>> >
Reading symbols from <br>>/lib/tls/libc.so.6...done.<br>>
> Loaded symbols for <br>>/lib/tls/libc.so.6<br>>
> Reading symbols from
<br>>/lib/ld-linux.so.2...done.<br>> > Loaded symbols
for <br>>/lib/ld-linux.so.2<br>> > #0 0x00df4cef in
raise () from <br>>/lib/tls/libc.so.6<br>> > (gdb)
bt<br>> > #0 0x00df4cef <br>>in raise () from
/lib/tls/libc.so.6<br>> > #1 0x00df64f5 in
<br>>abort () from /lib/tls/libc.so.6<br>> > #2
0x08054126 in <br>>sigsegv_handler (signum=11) at
sig.c:57<br>> > #3 <signal <br>>handler
called><br>> > #4 0x00e46cac in mempcpy () from
<br>>/lib/tls/libc.so.6<br>> > #5 0x00e3a4d2 in
<br>>_IO_default_xsputn_internal () from
/lib/tls/libc.so.6<br>> > <br>>#6 0x00e13527 in
vfprintf () from /lib/tls/libc.so.6<br>> > <br>>#7
0x00e2f3dc in vsprintf () from /lib/tls/libc.so.6<br>> >
<br>>#8 0x00e1a03d in sprintf () from
/lib/tls/libc.so.6<br>> > #9 <br>> 0x0804d7a4 in
do_message (msg=0x9e0b3f8, origin=0x80554bb <br>>"")
at<br>> > hobbitd.c:1903<br>> > #10
<br>>0x0804fcb5 in main (argc=8, argv=0xbfff9084) at
<br>>hobbitd.c:2944<br>> > (gdb)<br>>
><br>> > Now I <br>>try to find out which of
the new hosts - and what test causes <br>>the<br>> >
problems...<br>> ><br>> >
<br>>Regards,<br>> ><br>> >
Stefan Loos<br>> ><br>>
<br>>><br>> ><br>> > To
unsubscribe from the hobbit list, <br>>send an e-mail
to<br>> > hobbit-unsubscribe at hswn.dk<br>>
<br>>><br>>
><br>><br>>--<br>>Henrik
<br>>Storner<br>><br>>To unsubscribe from the
hobbit list, send an <br>>e-mail
to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br><br>><br>><br>><br>>To
unsubscribe from the hobbit list, send an e-mail
to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>
On Thu, May 12, 2005 at 12:31:44PM +0000, Stefan Loos wrote:
is the number of tests per host limited? I've cleaned my bb-hosts and as I add one host with many tests. I disabled all customized tests and tried to find out if its a single test. But all test run by itself didn't crash the hobbit server - running all together did! The host runs about 20 tests.
No, there is no limit (other than running out of memory, but I think we can rule out that one).
It seems to crash while loading the checkpoint file. Could you send me that file - it's the ~hobbit/server/tmp/hobbitd.chk file ? I believe it has somehow become corrupted, but that still shouldn't crash the server.
Regards, Henrik
participants (2)
-
henrik＠hswn.dk
-
stefan_loos＠hotmail.com