Hobbit 4.3 running on VMWare causes interesting hanging...
Hello,
since a long time, I run hobbit on a VM-Ware cluster. Works very well in the past. However, the serverguys have updated the VM-Ware to 3.something and also reinstall the VM-Ware drivers on the Suse Linux 10.1...
Since a while now, hobbit stops working several times a day... The system looks like this:
ps aux...
root 3225 0.0 0.1 1960 636 tty6 Ss+ 11:32 0:00 /sbin/mingetty tty6 hobbit 3256 0.0 0.1 1836 548 ? S 11:32 0:00 hobbitd_channel --channel=stachg --log=/var/log/hobbit/history.log hobbit 3257 0.0 0.1 2092 524 ? S 11:32 0:00 hobbitd_channel --channel=clichg --log=/var/log/hobbit/hostdata.log hobbit 3258 0.0 0.0 1832 464 ? S 11:32 0:00 hobbitd_channel --channel=notes --log=/var/log/hobbit/notes.log hob hobbit 3259 0.0 0.1 1836 548 ? S 11:32 0:00 hobbitd_channel --channel=page --log=/var/log/hobbit/page.log hobbi hobbit 3260 0.0 0.1 1836 548 ? S 11:32 0:01 hobbitd_channel --channel=status --log=/var/log/hobbit/rrd-status.l hobbit 3261 0.0 0.0 1832 468 ? S 11:32 0:00 hobbitd_channel --channel=data --log=/var/log/hobbit/rrd-data.log h hobbit 3262 0.0 0.1 2088 540 ? S 11:32 0:00 hobbitd_channel --channel=client --log=/var/log/hobbit/clientdata.l hobbit 3274 0.0 0.3 3772 1736 ? S 11:32 0:02 hobbitd_rrd --rrddir=/home/hobbit/data/rrd hobbit 3275 0.0 0.2 2512 1504 ? S 11:32 0:00 hobbitd_client hobbit 3276 0.0 0.1 1856 848 ? S 11:32 0:00 hobbitd_history hobbit 3277 0.0 0.2 2260 1208 ? S 11:32 0:00 hobbitd_alert --checkpoint-file=/home/hobbit/server/tmp/alert.chk - hobbit 3278 0.0 0.1 2104 560 ? S 11:32 0:00 hobbitd_hostdata hobbit 3279 0.0 0.2 3644 1540 ? S 11:32 0:00 hobbitd_rrd --rrddir=/home/hobbit/data/rrd hobbit 3280 0.0 0.0 2004 440 ? S 11:32 0:00 hobbitd_filestore --notes root 9923 0.0 0.4 8416 2396 ? Ss 12:47 0:00 sshd: root at pts/1 root 9933 0.0 0.3 4132 1872 pts/1 Ss+ 12:47 0:00 -bash wwwrun 13515 0.0 2.0 62104 10588 ? S 14:16 0:00 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -DSSL root 14169 0.0 0.1 2388 844 pts/1 R+ 14:32 0:00 ps aux
server:~# rchobbit stop /home/hobbit/server/bin/hobbit.sh: line 44: kill: (3155) - No such process Hobbit stopped
I have to manually kill all other hobbit processes, before a restart is successful.
Does anybody have an idea, where I should look for?
mit freundlichen Grüßen - Best regards Hälsningar - Met vriendelijke groeten Sincères salutations
Oliver Grube Campbells Germany GmbH Registergericht Luebeck Reg. Nr. HRB 4082 Geschaeftsfuehrer: Joseph B. Folds III
Geschaeftssitz: Geniner Strasse 88 - 100 23560 Luebeck Deutschland
This e-mail and any files transmitted with it may contain confidential information and is intended solely for use by the individual to whom it is addressed. If you received this e-mail in error, please notify the sender, do not disclose its contents to others and delete it from your system.
On Mon, Nov 26, 2007 at 02:38:16PM +0100, Oliver Grube wrote:
since a long time, I run hobbit on a VM-Ware cluster. Works very well in the past. However, the serverguys have updated the VM-Ware to 3.something and also reinstall the VM-Ware drivers on the Suse Linux 10.1...
Since a while now, hobbit stops working several times a day... The system looks like this:
ps aux...
Not easy to read. Could you show me a "ps f -u hobbit" or "pstree -a -u hobbit", please ?
As far as I can tell, there is not "hobbitlaunch" or "hobbitd" process running. Have you checked the hobbitd.log and hobbitlaunch.log files?
Henrik
Hi Henrik!
Thanks for your fast response!
Unfortunately, they have rebooted the server in the meantime and hobbit looks like running again - or not? This is, what you have requested:
server:~ # pstree -a -u hobbit hobbitlaunch --config=/home/hobbit/server/etc/hobbitlaunch.cfg--env=/home/hobbit/server/etc/hobbitserver.cf +-bbtest-net --report --ping --checkresponse ¦ +-sh -c traceroute\04010.170.1.254\0402>&1 ¦ +-traceroute 10.170.1.254 +-hobbitd --pidfile=/var/log/hobbit/hobbitd.pid --restart=/home/hobbit/server/tmp/hobbitd.chk... +-hobbitd_channel --channel=stachg --log=/var/log/hobbit/history.log hobbitd_history ¦ +-hobbitd_history +-hobbitd_channel --channel=clichg --log=/var/log/hobbit/hostdata.log hobbitd_hostdata ¦ +-hobbitd_hostdat +-hobbitd_channel --channel=notes --log=/var/log/hobbit/notes.log hobbitd_filestore --notes ¦ +-hobbitd_filesto --notes +-hobbitd_channel --channel=page --log=/var/log/hobbit/page.log hobbitd_alert ... ¦ +-hobbitd_alert --checkpoint-file=/home/hobbit/server/tmp/alert.chk --checkpoint-interval=600 +-hobbitd_channel --channel=status --log=/var/log/hobbit/rrd-status.log hobbitd_rrd --rrddir=/home/hobbit/data/rrd ¦ +-hobbitd_rrd --rrddir=/home/hobbit/data/rrd +-hobbitd_channel --channel=data --log=/var/log/hobbit/rrd-data.log hobbitd_rrd --rrddir=/home/hobbit/data/rrd ¦ +-hobbitd_rrd --rrddir=/home/hobbit/data/rrd +-hobbitd_channel --channel=client --log=/var/log/hobbit/clientdata.log hobbitd_client +-hobbitd_client sh -c... +-vmstat 300 2 sh -c... +-vmstat 300 2 sh -c... +-vmstat 300 2 sh -c... +-vmstat 300 2 sh -c... +-vmstat 300 2 server:~ #
hobbitd.log => shows only "Setup complete" entries everytime, we restarted the server. Nothing else, beside some "Oversize page/ack/notify msg " and "Oversize status msg from" which didn't (by the date) seams to be related. 2007-11-26 15:01:40 Setup complete
hobbitlaunch.log
=> There is something interesting: 007-11-26 11:32:50 Setting up hobbitd channels 2007-11-26 11:32:50 Setting up logfiles 2007-11-26 14:58:44 hobbitlaunch starting 2007-11-26 14:58:44 Loading tasklist configuration from /home/hobbit/server/etc/hobbitlaunch.cfg 2007-11-26 14:58:44 Loading hostnames 2007-11-26 14:58:44 Loading saved state 2007-11-26 14:58:44 Setting up network listener on 0.0.0.0:1984 2007-11-26 14:58:44 Setting up local listener 2007-11-26 14:58:44 Setting up signal handlers 2007-11-26 14:58:44 Setting up hobbitd channels 2007-11-26 14:58:44 FATAL: hobbitd sees clientcount 1, should be 0 Check for hanging hobbitd_channel processes or stale semaphores 2007-11-26 14:58:44 Cannot setup status channel 2007-11-26 14:58:44 Task hobbitd terminated, status 1
And this is straight after a reboot of the server...
A misconfiguration? What did I wrong?
mit freundlichen Grüßen - Best regards Hälsningar - Met vriendelijke groeten Sincères salutations
Oliver Grube EU Security&Controls Manager
Campbellsoup Company Tel.: +49 451/ 5306-226 Fax: +49 451/ 5306-356
Henrik Stoerner
<henrik at hswn.dk>
To
2007-11-26 15:08 hobbit at hswn.dk
cc
Please respond to Subject
hobbit at hswn.dk Re: [hobbit] Hobbit 4.3 running on
VMWare causes interesting
hanging...
On Mon, Nov 26, 2007 at 02:38:16PM +0100, Oliver Grube wrote:
since a long time, I run hobbit on a VM-Ware cluster. Works very well in the past. However, the serverguys have updated the VM-Ware to 3.something and also reinstall the VM-Ware drivers on the Suse Linux 10.1...
Since a while now, hobbit stops working several times a day... The system looks like this:
ps aux...
Not easy to read. Could you show me a "ps f -u hobbit" or "pstree -a -u hobbit", please ?
As far as I can tell, there is not "hobbitlaunch" or "hobbitd" process running. Have you checked the hobbitd.log and hobbitlaunch.log files?
Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Campbells Germany GmbH Registergericht Luebeck Reg. Nr. HRB 4082 Geschaeftsfuehrer: Joseph B. Folds III
Geschaeftssitz: Geniner Strasse 88 - 100 23560 Luebeck Deutschland
This e-mail and any files transmitted with it may contain confidential information and is intended solely for use by the individual to whom it is addressed. If you received this e-mail in error, please notify the sender, do not disclose its contents to others and delete it from your system.
participants (2)
-
henrik＠hswn.dk
-
oliver_grube＠campbellsoup.com