After Server Restart get an error and all tests are purple
Hi, After Server Restart i get an error and all tests are purple. (except conn, http)
i get the following error: 2007-01-18 08:28:37 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:28:39 Could not connect to bbd at xxx.de:1984 - Connection refused
what can i do?
thx Stefan
Hi Stefan,
please give some more information. Restart of hobbit server or reboot of the machine? Is the client running (it should start automatically while restarting the server but this doesn't work properly)? Are tests from other machines concerned or only local tests on the server? What do the client logs say?
greetings Rolf
Hi, After Server Restart i get an error and all tests are purple. (except conn, http)
i get the following error: 2007-01-18 08:28:37 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:28:39 Could not connect to bbd at xxx.de:1984 - Connection refused
what can i do?
thx Stefan
-- Mit freundlichen Gruessen Rolf Schrittenlocher
HRZ/BDV, Senckenberganlage 31, 60054 Frankfurt Tel: (49) 69 - 798 28908 Fax: (49) 69 798 2881 LBS: lbs-f at mlist.uni-frankfurt.de Persoenlich: schritte at rz.uni-frankfurt.de
Hi Stefan,
Hi
please give some more information. Restart of hobbit server or reboot of
the machine?
Of the machine. But if i try to restart hobbit server there is no other result.
Is the client running (it should start automatically while restarting the server but this doesn't work properly)? Yes it runs.
Are tests from other machines concerned or only local tests on the server?
No all other systems can send tests.
What do the client logs say?
cat clientlaunch.log|tail -n 30 2006-11-22 13:34:21 hobbitlaunch starting 2006-11-22 13:34:21 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2006-11-22 16:32:47 hobbitlaunch starting 2006-11-22 16:32:47 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 13:51:12 hobbitlaunch starting 2007-01-17 13:51:12 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 15:19:16 hobbitlaunch starting 2007-01-17 15:19:16 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 16:24:23 hobbitlaunch starting 2007-01-17 16:24:23 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 16:26:37 hobbitlaunch starting 2007-01-17 16:26:37 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 16:37:11 hobbitlaunch starting 2007-01-17 16:37:11 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-18 08:28:36 hobbitlaunch starting 2007-01-18 08:28:36 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg clientlaunch.log
cat hobbitclient.log|tail -n 30 2007-01-18 08:53:57 Unknown token 'localhost:' ignored at line 351 2007-01-18 08:54:00 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:00 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:03 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:03 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:06 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:06 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:09 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:09 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:09 Failed to get a message, terminating 2007-01-18 08:59:01 Unknown token 'localhost:' ignored at line 351 2007-01-18 08:59:04 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:04 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:07 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:07 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:10 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:10 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:13 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:13 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:13 Failed to get a message, terminating 2007-01-18 09:04:04 Unknown token 'localhost:' ignored at line 351 2007-01-18 09:04:07 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:07 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:10 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:10 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:13 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:13 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:16 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:16 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:16 Failed to get a message, terminating
cu Stefan
Hi Stefan,
so I suppose your client is gathering data correctly (in the client/tmp directory) but cannot send it. What is striking me is the line: Unknown token 'localhost:' ignored at line 351 As all other clients can send to your server it seems that only the server's client is missing something. But unfortunately I have no idea what it could be. Maybe you should grep for localhost in the client-directories.
regards Rolf
Hi Stefan,
Hi
please give some more information. Restart of hobbit server or reboot of the machine?
Of the machine. But if i try to restart hobbit server there is no other result.
Is the client running (it should start automatically while restarting the server but this doesn't work properly)? Yes it runs.
Are tests from other machines concerned or only local tests on the server?
No all other systems can send tests.
What do the client logs say?
cat clientlaunch.log|tail -n 30 2006-11-22 13:34:21 hobbitlaunch starting 2006-11-22 13:34:21 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2006-11-22 16:32:47 hobbitlaunch starting 2006-11-22 16:32:47 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 13:51:12 hobbitlaunch starting 2007-01-17 13:51:12 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 15:19:16 hobbitlaunch starting 2007-01-17 15:19:16 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 16:24:23 hobbitlaunch starting 2007-01-17 16:24:23 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 16:26:37 hobbitlaunch starting 2007-01-17 16:26:37 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-17 16:37:11 hobbitlaunch starting 2007-01-17 16:37:11 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg 2007-01-18 08:28:36 hobbitlaunch starting 2007-01-18 08:28:36 Loading tasklist configuration from /opt/hobbit/current/client/etc/clientlaunch.cfg clientlaunch.log
cat hobbitclient.log|tail -n 30 2007-01-18 08:53:57 Unknown token 'localhost:' ignored at line 351 2007-01-18 08:54:00 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:00 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:03 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:03 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:06 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:06 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:09 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:54:09 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:54:09 Failed to get a message, terminating 2007-01-18 08:59:01 Unknown token 'localhost:' ignored at line 351 2007-01-18 08:59:04 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:04 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:07 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:07 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:10 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:10 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:13 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 08:59:13 Whoops ! bb failed to send message - Connection failed 2007-01-18 08:59:13 Failed to get a message, terminating 2007-01-18 09:04:04 Unknown token 'localhost:' ignored at line 351 2007-01-18 09:04:07 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:07 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:10 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:10 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:13 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:13 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:16 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 09:04:16 Whoops ! bb failed to send message - Connection failed 2007-01-18 09:04:16 Failed to get a message, terminating
cu Stefan
-- Mit freundlichen Gruessen Rolf Schrittenlocher
HRZ/BDV, Senckenberganlage 31, 60054 Frankfurt Tel: (49) 69 - 798 28908 Fax: (49) 69 798 2881 LBS: lbs-f at mlist.uni-frankfurt.de Persoenlich: schritte at rz.uni-frankfurt.de
Hi, has noone an Idea? Here two more logs:
hobbitlaunch.log: 2007-01-18 13:46:37 hobbitlaunch starting 2007-01-18 13:46:37 Loading tasklist configuration from /opt/hobbit/current/server/etc/hobbitlaunch.cfg 2007-01-18 13:46:38 Loading hostnames 2007-01-18 13:46:38 Loading saved state 2007-01-18 13:46:38 Setting up network listener on 0.0.0.0:1984 2007-01-18 13:46:38 Setting up signal handlers 2007-01-18 13:46:38 Setting up hobbitd channels 2007-01-18 13:46:38 Setting up logfiles
bb-display.log 2006-11-15 16:13:20 Cannot create file Test///Test.html.tmp (in /var/www/html/hobbit): Too many open files in system 2006-11-15 16:13:20 Cannot create file bb2.html.tmp: Too many open files in system 2006-11-15 16:13:20 Cannot create file bbnk.html.tmp: Too many open files in system 2006-11-15 16:13:20 Whoops ! bb failed to send message - Cannot get a socket
mfg Stefan
stefan.freisler at aspect-online.de schrieb am 18.01.2007 08:35:55:
Hi, After Server Restart i get an error and all tests are purple. ( except conn, http)
i get the following error: 2007-01-18 08:28:37 Whoops ! bb failed to send message - Connection
failed
2007-01-18 08:28:39 Could not connect to bbd at xxx.de:1984 - Connection refused
what can i do?
thx Stefan
I think the bb-display.log file has your answer; the system can't open a socket connection to the BB (hobbit) server. Is some process running away with using up server resources? In our environment under Linux, we've had issues with auditd filling up the process table; maybe the same thing is happening on your machine? For us, this happens when /var fills up about 80 or 90% (I forget the exact number).
Hi, has noone an Idea? Here two more logs:
hobbitlaunch.log: 2007-01-18 13:46:37 hobbitlaunch starting 2007-01-18 13:46:37 Loading tasklist configuration from /opt/hobbit/current/server/etc/hobbitlaunch.cfg 2007-01-18 13:46:38 Loading hostnames 2007-01-18 13:46:38 Loading saved state 2007-01-18 13:46:38 Setting up network listener on 0.0.0.0:1984 2007-01-18 13:46:38 Setting up signal handlers 2007-01-18 13:46:38 Setting up hobbitd channels 2007-01-18 13:46:38 Setting up logfiles
bb-display.log 2006-11-15 16:13:20 Cannot create file Test///Test.html.tmp (in /var/www/html/hobbit): Too many open files in system 2006-11-15 16:13:20 Cannot create file bb2.html.tmp: Too many open files in system 2006-11-15 16:13:20 Cannot create file bbnk.html.tmp: Too many open files in system 2006-11-15 16:13:20 Whoops ! bb failed to send message - Cannot get a socket
mfg Stefan
stefan.freisler at aspect-online.de schrieb am 18.01.2007 08:35:55:
Hi, After Server Restart i get an error and all tests are purple. ( except conn, http)
i get the following error: 2007-01-18 08:28:37 Whoops ! bb failed to send message - Connection
failed
2007-01-18 08:28:39 Could not connect to bbd at xxx.de:1984 - Connection refused
what can i do?
thx Stefan
looks like a "ulimit" problem -- you need to permit more concurrent open filess. This could be a symptom of another problem -- like files not getting closed when they should.
GLH
From: stefan.freisler at aspect-online.de[mailto:stefan.freisler at aspect-online.de] Sent: Thursday, January 18, 2007 8:15 AM To: hobbit at hswn.dk Subject: [hobbit] Antwort: [hobbit] After Server Restart get an error and all tests are purple
Hi,
has noone an Idea? Here two more logs:
hobbitlaunch.log:
2007-01-18 13:46:37 hobbitlaunch starting
2007-01-18 13:46:37 Loading tasklist configuration from/opt/hobbit/current/server/etc/hobbitlaunch.cfg 2007-01-18 13:46:38 Loading hostnames 2007-01-18 13:46:38 Loading saved state 2007-01-18 13:46:38 Setting up network listener on 0.0.0.0:1984 2007-01-18 13:46:38 Setting up signal handlers 2007-01-18 13:46:38 Setting up hobbitd channels 2007-01-18 13:46:38 Setting up logfiles
bb-display.log
2006-11-15 16:13:20 Cannot create file Test///Test.html.tmp (in/var/www/html/hobbit): Too many open files in system 2006-11-15 16:13:20 Cannot create file bb2.html.tmp: Too many open files in system 2006-11-15 16:13:20 Cannot create file bbnk.html.tmp: Too many open files in system 2006-11-15 16:13:20 Whoops ! bb failed to send message - Cannot get a socket
mfg
Stefan
stefan.freisler at aspect-online.de schrieb am 18.01.2007 08:35:55:
>
> Hi,
> After Server Restart i get an error and all tests are purple.( > except conn, http) > > i get the following error: > 2007-01-18 08:28:37 Whoops ! bb failed to send message - Connection failed > 2007-01-18 08:28:39 Could not connect to bbd at xxx.de:1984 - Connection refused > > what can i do? > > thx > Stefan
Hi, oh sorry! I see this log file was from November sorry my fault.
But i have an other thing:
/opt/hobbit/current/client/bin/bb --debug 10.28.1.10 "status bla,blab,bla,de.msgs gree Only a test" 2007-01-18 16:32:00 Transport setup is: 2007-01-18 16:32:00 bbdportnumber = 1984 2007-01-18 16:32:00 bbdispproxyhost = NONE 2007-01-18 16:32:00 bbdispproxyport = 0 2007-01-18 16:32:00 Recipient listed as '10.28.1.10' 2007-01-18 16:32:00 Standard BB protocol on port 1984 2007-01-18 16:32:00 Will connect to address 10.28.1.10 port 1984 2007-01-18 16:32:03 Connect status is 111 2007-01-18 16:32:03 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 16:32:03 Whoops ! bb failed to send message - Connection failed
Can i somewhere read what status 111 means?
thx a lot
mfg Stefan
Have you checked that the deamon is listening on port 1984 on the server 10.28.1.10 ? next thing could be to telnet from the client to the server on the port to check if any firewall/routing issues are not in the way.
stefan.freisler at aspect-online.de wrote:
Hi, oh sorry! I see this log file was from November sorry my fault.
But i have an other thing:
/opt/hobbit/current/client/bin/bb --debug 10.28.1.10 "status bla,blab,bla,de.msgs gree Only a test" 2007-01-18 16:32:00 Transport setup is: 2007-01-18 16:32:00 bbdportnumber = 1984 2007-01-18 16:32:00 bbdispproxyhost = NONE 2007-01-18 16:32:00 bbdispproxyport = 0 2007-01-18 16:32:00 Recipient listed as '10.28.1.10' 2007-01-18 16:32:00 Standard BB protocol on port 1984 2007-01-18 16:32:00 Will connect to address 10.28.1.10 port 1984 2007-01-18 16:32:03 Connect status is 111 2007-01-18 16:32:03 Could not connect to bbd at 10.28.1.10:1984 - Connection refused 2007-01-18 16:32:03 Whoops ! bb failed to send message - Connection failed
Can i somewhere read what status 111 means?
thx a lot
mfg Stefan
participants (5)
-
greg.hubbard＠eds.com
-
gumby3203＠gmail.com
-
Schrittenlocher＠rz.uni-frankfurt.de
-
stefan.freisler＠aspect-online.de
-
tlp-hobbit＠holme-pedersen.dk