Yes. Quite often. ---snip--- 2005-07-04 14:09:17 Whoops ! bb failed to send message - timeout 2005-07-04 14:09:17 Could not get the Hobbit statuslog-list 2005-07-04 14:09:50 Whoops ! bb failed to send message - timeout 2005-07-04 14:09:50 hobbitd status-board not available 2005-07-04 14:10:49 Whoops ! bb failed to send message - timeout 2005-07-04 14:10:49 hobbitd status-board not available 2005-07-04 14:11:49 Whoops ! bb failed to send message - timeout 2005-07-04 14:11:49 hobbitd status-board not available 2005-07-04 14:12:52 Whoops ! bb failed to send message - timeout 2005-07-04 14:12:52 hobbitd status-board not available 2005-07-04 14:13:50 Whoops ! bb failed to send message - timeout 2005-07-04 14:13:50 hobbitd status-board not available 2005-07-04 14:14:50 Whoops ! bb failed to send message - timeout 2005-07-04 14:14:50 hobbitd status-board not available 2005-07-04 14:16:22 Whoops ! bb failed to send message - timeout 2005-07-04 14:16:22 hobbitd status-board not available 2005-07-04 14:16:22 WARNING: Runtime 61 longer than BBSLEEP (60) 2005-07-04 14:16:52 Whoops ! bb failed to send message - timeout 2005-07-04 14:16:52 hobbitd status-board not available 2005-07-04 14:17:52 Whoops ! bb failed to send message - timeout 2005-07-04 14:17:52 hobbitd status-board not available 2005-07-04 14:18:52 Whoops ! bb failed to send message - timeout 2005-07-04 14:18:52 hobbitd status-board not available 2005-07-04 14:19:52 Whoops ! bb failed to send message - timeout 2005-07-04 14:19:52 hobbitd status-board not available 2005-07-04 14:21:26 Whoops ! bb failed to send message - timeout 2005-07-04 14:21:26 hobbitd status-board not available 2005-07-04 14:21:26 WARNING: Runtime 61 longer than BBSLEEP (60) 2005-07-04 14:21:59 Whoops ! bb failed to send message - timeout 2005-07-04 14:21:59 hobbitd status-board not available ---snip---
-----Original Message----- From: Stefan Loos [mailto:stefan_loos at hotmail.com] Sent: Monday, 4 July 2005 2:16 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Status Unavailable
Hello Vernon,
can you tell me, if there is anything like "hobbitd status board not available" in the bb-display.log?
Regards,
Stefan
<br><br><br>>From: "Vernon Everett" <v.everett at afgonline.com.au><br>>Reply-To: hobbit at hswn.dk<br>>To: <hobbit at hswn.dk><br>>Subject: RE: [hobbit] Status Unavailable<br>>Date: Fri, 1 Jul 2005 16:56:38 +0800<br>><br>>Hi Henrik<br>><br>>It should be idle. All the system does is run hobbit. :-)<br>><br>>Hobbitd is currently dead in the water.<br>> [root at pengo log]# strace -p 3025<br>> Process 3025 attached - interrupt to quit<br>> futex(0x40141b20, FUTEX_WAIT, 2,
NULL<br>><br>>And it's been like this a while.<br>>When I did the kill -6 I got this.<br>> [root at pengo log]# strace -p 3025<br>> Process 3025 attached - interrupt to quit<br>> futex(0x40141b20, FUTEX_WAIT, 2, NULL) = -1 EINTR (Interrupted<br>>system call)<br>> --- SIGABRT (Aborted) @ 0 (0) ---<br>> Process 3025 detached<br>>Which I suppose was expected :-)<br>><br>>I restarted it, and got this.<br>> [root at pengo etc]# strace -p 9223<br>> Process 9223 attached
- interrupt to quit<br>> semop(32769, 0xbfffe3a0, 1<br>>Nope, there is nothing I forgot to cut and paste.<br>>That really was it.<br>><br>>And this shit just gets stranger and stranger.<br>>It isn't dumping core.<br>>I hit it with a kill -6 and nothing happens.<br>>I then thought maybe we were both mistaken, and had the command wrong or<br>>my linux was defaulted to not core, so I started vi in a session and did<br>>a kill -6 on that. That dumped?!<br>>Hobbit isn't dumping.<br>><br>>I rebooted and tried again.<br>>I managed to get a nice strace output - see attached
- but still no damn<br>>core.<br>><br>>OK, I added debug, and
restarted.<br>>When I went to check the logs, I found this in
hobbitlaunch.log.<br>>---snip---<br>>2005-07-01 16:37:21 Loading
tasklist configuration
from<br>>/usr/lib/hobbit/server/etc/hobbitlaunch.cfg<br>>2005-07-0
1
16:37:21 Loading hostnames<br>>2005-07-01 16:37:21 Loading saved
state<br>>2005-07-01 16:37:21 Setting up network listener on
0.0.0.0:1984<br>>2005-07-01 16:37:21 Cannot bind to listen socket
(Address already in<br>>use)<br>>2005-07-01 16:37:21 Task hobbitd
started with PID 4761<br>>2005-07-01 16:37:26 Task hobbitd
terminated, status 1<br>>2005-07-01 16:37:26 Loading
hostnames<br>>2005-07-01
16:37:26 Loading saved state<br>>2005-07-01 16:37:26 Task hobbitd
started with PID 4765<br>>2005-07-01 16:37:26 Setting up network
listener on
0.0.0.0:1984<br>>2005-07-01 16:37:26 Cannot bind to listen socket
(Address already in<br>>use)<br>>2005-07-01 16:37:26 Task hobbitd
terminated, status 1<br>>2005-07-01 16:37:31 Loading
hostnames<br>>2005-07-01 16:37:31 Loading saved
state<br>>2005-07-01
16:37:31 Task hobbitd started with PID 4770<br>>2005-07-01 16:37:31
Setting up network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:31
Cannot bind to listen socket (Address already
in<br>>use)<br>>2005-07-01 16:37:31 Task hobbitd terminated,
status
1<br>>2005-07-01 16:37:36 Task hobbitd started with PID
4774<br>>2005-07-01 16:37:36 Loading hostnames<br>>2005-07-01
16:37:36 Loading saved state<br>>2005-07-01 16:37:36 Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:36 Cannot bind
to listen socket (Address already in<br>>use)<br>>2005-07-01
16:37:36 Task hobbitd terminated, status 1<br>>2005-07-01 16:37:41
Task hobbitd started with PID 4778<br>>2005-07-01 16:37:41 Loading
hostnames<br>>2005-07-01
16:37:41 Loading saved state<br>>2005-07-01 16:37:41 Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:41 Cannot bind
to listen socket (Address already in<br>>use)<br>>2005-07-01
16:37:41 Task hobbitd terminated, status 1<br>>2005-07-01 16:37:46
Task hobbitd started with PID 4783<br>>2005-07-01 16:37:46 Loading
hostnames<br>>2005-07-01
16:37:46 Loading saved state<br>>2005-07-01 16:37:46 Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:46 Cannot bind
to listen socket (Address already in<br>>use)<br>>2005-07-01
16:37:46 Task hobbitd terminated, status
1<br>>---snip---<br>><br>>Looks like a clue.<br>>I will add
the output of netstat -a<br>><br>>Got the hobbitd.log file for you
too.<br>><br>>Let me know if there is
anything else I can get you.<br>><br>>Regards<br>>
Vernon<br>><br>>P.S. Your cold one is quickly becoming many cold ones if you ever get to<br>>Perth<br>><br>><br>><br>><br>><br>>-----Orig inal Message-----<br>>From: Henrik Stoerner [mailto:henrik at hswn.dk]<br>>Sent: Friday, 1 July 2005 3:38 PM<br>>To: hobbit at hswn.dk<br>>Subject: Re: [hobbit] Status Unavailable<br>><br>>On Fri, Jul 01, 2005 at 03:25:30PM +0800, Vernon Everett wrote:<br>> > Thanks for helping on this.<br>> > I rebooted this morning. Could the memory leak still effect me in that<br>><br>> > short time?<br>><br>>Probably not. Just wanted to rule out this possibility.<br>><br>> > No "failed allocation" in dmesg output.<br>> > Do you want the full output?<br>><br>>No, I dont think that is necessary.<br>><br>> > [root at pengo log]# vmstat 4 20<br>><br>>And your system is mostly idle with no swap or disk activity.<br>><br>> > [hobbit at pengo hobbit]$ server/bin/bb 127.0.0.1 "hobbitdboard"<br>> > 2005-07-01 15:21:45 Whoops ! bb failed to send message - timeout<br>><br>>Could you try running "strace -p <process-ID of the hobbitd process>"<br>>for a minute or two and send me the output, then do a "kill -6<br>><process-id>" and mail me the core-file from ~hobbit/server/tmp/<br>>together with the ~hobbit/server/bin/hobbitd file ?<br>><br>>Also, after this try adding a "--debug" to the hobbitd commandline in<br>>hobbitlaunch.cfg. Let it run for a while and then mail me the<br>>hobbitd.log file.<br>><br>>This bug sounds a bit nasty, I think ....<br>><br>><br>>Regards,<br>>Henrik<br>><br>><br>&g t;To unsubscribe from the hobbit list, send an e-mail to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>>_ _ _ _ _ _
_<br>><br>>NOTICE: This message and any attachments are confidential and may contain copyright material<br>>of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the<br>>addressee and any other named recipient. If you are not the intended recipient, any use,<br>>distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached<br>>to this message is not waived or lost by reason of the mistaken transmission or delivery to any<br>>unintended party. If you have received this message in error, please notify the author immediately or<br>>contact Australian Finance Group on +61 8 9420 7888.<br>><br>><br>>To unsubscribe from the hobbit list, send an e-mail to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
NOTICE: This message and any attachments are confidential and may contain copyright material of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the addressee and any other named recipient. If you are not the intended recipient, any use, distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached to this message is not waived or lost by reason of the mistaken transmission or delivery to any unintended party. If you have received this message in error, please notify the author immediately or contact Australian Finance Group on +61 8 9420 7888.