August 2006 - Xymon - Xymon mailman web

[hobbit] AIX 5.2 clientdata problem (was: Note: Hobbit 4.2 re lease planned for Thursday Aug. 1 0.)
by CHRIS.MORRIS＠RWEnpower.com 09 Aug '06

09 Aug '06

Hi Henrik, I am running the snapshot from 7th August on this server, with the pscheck patch applied. You resolved the clientdata problem on 12th July. The hostdata module does not terminate, but just keeps saying "Our child has failed and will not talk to us: Channel clichg, PID 38436" My hobbitlaunch.cfg file contains the following lines :- [hobbitd] HEARTBEAT ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg CMD hobbitd --pidfile=$BBSERVERLOGS/hobbitd.pid --restart=$BBTMP/hobbitd.chk --checkpoint-file=$BBTMP/hobbitd.chk --checkpoint-interval=600 --log=$BBSERVERLOGS/ho bbitd.log --store-clientlogs=!msgs --admin-senders=127.0.0.1,$BBSERVERIP . . # Note: The --store-clientlogs option for the [hobbitd] provides control over # which status-changes will cause a client message to be stored. [hostdata] ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg NEEDS hobbitd CMD hobbitd_channel --channel=clichg --debug --log=$BBSERVERLOGS/hostdata.log --logdir=$BBVAR/hostdata hobbitd_hostdata . . The hobbitlaunch.log file :- 2006-08-09 10:06:29 Loading tasklist configuration from /usr/local/hobbit/server/etc/hobbitlaunch.cfg 2006-08-09 10:06:47 hobbitlaunch starting 2006-08-09 10:06:47 Loading tasklist configuration from /usr/local/hobbit/server/etc/hobbitlaunch.cfg 2006-08-09 10:06:47 Loading hostnames 2006-08-09 10:06:47 Loading saved state 2006-08-09 10:06:47 Setting up network listener on 0.0.0.0:1984 2006-08-09 10:06:47 Setting up signal handlers 2006-08-09 10:06:47 Setting up hobbitd channels 2006-08-09 10:06:47 Setting up logfiles hobbit) ipcs -m IPC status from /dev/mem as of Wed 9 Aug 14:32:05 2006 T ID KEY MODE OWNER GROUP Shared Memory: m 0 0x580060c4 --rw-rw-rw- root system m 131073 0x52e74b4f --rw-rw-rw- imnadm imnadm m 131074 0x9308e451 --rw-rw-rw- imnadm imnadm m 131075 0xe4663d62 --rw-rw-rw- imnadm imnadm m 4 0xc76283cc --rw-rw-rw- imnadm imnadm m 5 0x298ee665 --rw-rw-rw- imnadm imnadm m 6 0xffffffff --rw-rw---- root system m 7 0x0d01aae1 --rw-rw-rw- root system m 3407880 0x0205b000 --rw------- hobbit hobbit m 4587529 0x0305b000 --rw------- hobbit hobbit m 13107210 0x4100a990 --rw-rw-rw- patrol adm m 4718603 0x0705b000 --rw------- hobbit hobbit m 4718604 0x0405b000 --rw------- hobbit hobbit m 4587533 0x0605b000 --rw------- hobbit hobbit m 4718606 0x0105b000 --rw------- hobbit hobbit m 4849679 0x0505b000 --rw------- hobbit hobbit m 3145744 0x0805b000 --rw------- hobbit hobbit m 2228241 0x0100a9a0 --rw-rw-rw- patrol adm m 1310738 0x0100a98b --rw-rw-rw- patrol adm m 32243731 0x4100a98c --rw-rw-rw- patrol adm m 9175060 0x4100a98d --rw-rw-rw- patrol adm m 1179669 0x4100a98f --rw-rw-rw- patrol adm m 2490390 0x4100a99a --rw-rw-rw- patrol adm m 1703959 0x4100a999 --rw-rw-rw- patrol adm m 1048600 0x4100a99b --rw-rw-rw- patrol adm m 13369369 0x4100a99f --rw-rw-rw- patrol adm m 11403290 0x4100a99d --rw-rw-rw- patrol adm m 12845083 0x4100a99e --rw-rw-rw- patrol adm m 26083356 0x4100a994 --rw-rw-rw- patrol adm The settings in hobbitserver.cfg are : MAXMSG_CLIENT="768" MAXMSG_STATUS="768" MAXMSG_STACHG="768" Regards, Chris > -----Original Message----- > From: henrik at hswn.dk [SMTP:henrik at hswn.dk] > Sent: Wednesday, August 09, 2006 1:51 PM > To: hobbit at hswn.dk > Subject: [hobbit] AIX 5.2 clientdata problem (was: Note: Hobbit 4.2 > release planned for Thursday Aug. 1 0.) > > (changing the subject) > > Hi Chris, > > it's quite a while ago - June 8, if my archive search is correct. > http://osiris.hswn.dk/hobbiton/2006/06/msg00203.html > > Have you tried running a current snapshot on this server? It looks > like it might be the same problem that You and I worked on for > quite some time between the beta-release and the RC-release - > the one where the clientdata task terminated without any apparent > reason. This mail started it: > http://osiris.hswn.dk/hobbiton/2006/06/msg00279.html > > If it's still an issue with the current snapshot, I'd like to see > the hobbitlaunch.log file also. And could you send me the output > from the "ipcs -m" command, please? > > > Regards, > Henrik > > > On Wed, Aug 09, 2006 at 10:25:31AM +0100, Morris, Chris (Shared Services) > wrote: > > Hi Henrik, > > > > A problem with hostdata that was raised some little while ago doesn't > seem > > to be resolved or listed as outstanding. > > > > When running the server on an AIX 5.2 system with the hostdata task > enabled, > > client data is not written to $BBVAR/hostdata and the hostdata.log > contains > > the following :- > > > > 2006-08-09 09:59:50 Our child has failed and will not talk to us: > Channel > > clichg, PID 17648 > > 2006-08-09 10:06:29 Setting up clichg channel (id=8) > > 2006-08-09 10:06:29 calling ftok('/usr/local/hobbit/server',8) > > 2006-08-09 10:06:29 ftok() returns: 0x805B000 > > 2006-08-09 10:06:29 shmget() returns: 0x46000E > > 2006-08-09 10:06:29 Setting up clichg channel (id=8) > > 2006-08-09 10:06:29 calling ftok('/usr/local/hobbit/server',8) > > 2006-08-09 10:06:29 ftok() returns: 0x805B000 > > 2006-08-09 10:06:29 shmget() returns: 0x46000E > > 2006-08-09 10:06:37 Semaphore wait aborted: Interrupted system call > > 2006-08-09 10:06:37 Semaphore wait aborted: Interrupted system call > > 2006-08-09 10:06:52 Setting up clichg channel (id=8) > > 2006-08-09 10:06:52 calling ftok('/usr/local/hobbit/server',8) > > 2006-08-09 10:06:52 ftok() returns: 0x805B000 > > 2006-08-09 10:06:52 shmget() returns: 0x300010 > > 2006-08-09 10:06:52 Setting up clichg channel (id=8) > > 2006-08-09 10:06:52 calling ftok('/usr/local/hobbit/server',8) > > 2006-08-09 10:06:52 ftok() returns: 0x805B000 > > 2006-08-09 10:06:52 shmget() returns: 0x300010 > > 2006-08-09 10:11:48 Our child has failed and will not talk to us: > Channel > > clichg, PID 38436 > > > > > > Looking forward to the 4.2 release - great work. > > > > > > Regards, > > > > Chris > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > **************************************************************************** The information contained in this email is intended only for the use of the intended recipient at the email address to which it has been addressed. If the reader of this message is not an intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination or copying of the message or associated attachments is strictly prohibited. If you have received this email in error, please contact the sender by return email or call 01793 877777 and ask for the sender and then delete it immediately from your system.Please note that neither RWE npower nor the sender accepts any responsibility for viruses and it is your responsibility to scan attachments (if any). *****************************************************************************

2 1

Questions on alerting
by shea_greg＠emc.com 09 Aug '06

09 Aug '06

Hi all, In an earlier post I had asked about alerting on each color (yellow or red) for disks and the consensus was to use the SCRIPT function and write something to do the alerting. So with parsing BBALPHAMSG and a couple of for loops, I can send a separate SNMP trap for each alert red, yellow, or a combination of both. Now comes the hard part, recovery. In order for me to clear the alerts in the MOM, I need to send a separate clear for each disk in error. I was looking to see if I could use the ACKCODE to correlate the errors with the clears but the clear sends a -1. So now some questions: How does the ACKCODE get generated ? Can the ACKCODE instead be -ACKCODE ? If so how ? Anyone else dealing with SNMP traps ? I am open for comments, suggestions, laughs Thanks -Grs- Gregory R Shea EMC Corporation

1 0

[hobbit] Note: Hobbit 4.2 release planned for Thursday Aug. 1 0.
by CHRIS.MORRIS＠RWEnpower.com 09 Aug '06

09 Aug '06

Hi Henrik, A problem with hostdata that was raised some little while ago doesn't seem to be resolved or listed as outstanding. When running the server on an AIX 5.2 system with the hostdata task enabled, client data is not written to $BBVAR/hostdata and the hostdata.log contains the following :- 2006-08-09 09:59:50 Our child has failed and will not talk to us: Channel clichg, PID 17648 2006-08-09 10:06:29 Setting up clichg channel (id=8) 2006-08-09 10:06:29 calling ftok('/usr/local/hobbit/server',8) 2006-08-09 10:06:29 ftok() returns: 0x805B000 2006-08-09 10:06:29 shmget() returns: 0x46000E 2006-08-09 10:06:29 Setting up clichg channel (id=8) 2006-08-09 10:06:29 calling ftok('/usr/local/hobbit/server',8) 2006-08-09 10:06:29 ftok() returns: 0x805B000 2006-08-09 10:06:29 shmget() returns: 0x46000E 2006-08-09 10:06:37 Semaphore wait aborted: Interrupted system call 2006-08-09 10:06:37 Semaphore wait aborted: Interrupted system call 2006-08-09 10:06:52 Setting up clichg channel (id=8) 2006-08-09 10:06:52 calling ftok('/usr/local/hobbit/server',8) 2006-08-09 10:06:52 ftok() returns: 0x805B000 2006-08-09 10:06:52 shmget() returns: 0x300010 2006-08-09 10:06:52 Setting up clichg channel (id=8) 2006-08-09 10:06:52 calling ftok('/usr/local/hobbit/server',8) 2006-08-09 10:06:52 ftok() returns: 0x805B000 2006-08-09 10:06:52 shmget() returns: 0x300010 2006-08-09 10:11:48 Our child has failed and will not talk to us: Channel clichg, PID 38436 Looking forward to the 4.2 release - great work. Regards, Chris > -----Original Message----- > From: henrik at hswn.dk [SMTP:henrik at hswn.dk] > Sent: Tuesday, August 08, 2006 10:22 PM > To: hobbit at hswn.dk > Subject: [hobbit] Note: Hobbit 4.2 release planned for Thursday Aug. > 10. > > Unless some major showstopper arrives within 24 hours, the 4.2 release > will be available Thursday morning around 8 AM UTC. > > In other words, this is your last chance to report bugs that must be > fixed before the final release. > > The "ps" handling bugs should all be fixed now in the latest snapshot > and all-in-one patch (dated 2006-08-08 21:13 UTC). > > A list of the bugs that have been fixed during the RC testing can be > found at the beta-patches page, http://www.hswn.dk/hobbitsw/betapatches/ > I would like to express my sincere admiration and thanks to the people > who have taken the trouble of testing Hobbit for this release; your > input and help in troubleshooting the problems has really made this > release a lot better than it was 4 weeks ago. > > > Thanks, > Henrik > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > **************************************************************************** The information contained in this email is intended only for the use of the intended recipient at the email address to which it has been addressed. If the reader of this message is not an intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination or copying of the message or associated attachments is strictly prohibited. If you have received this email in error, please contact the sender by return email or call 01793 877777 and ask for the sender and then delete it immediately from your system.Please note that neither RWE npower nor the sender accepts any responsibility for viruses and it is your responsibility to scan attachments (if any). *****************************************************************************

2 1

Hobbit on Solaris 10 x86
by Mike.Rowell＠Rightmove.co.uk 09 Aug '06

09 Aug '06

At the moment I'm seeing some rather strange things from clients and indeed a server running on solaris 10 x86. Occasionally and without pattern they will go purple, logging into the server and checking the logs indicate that there is no problem at all.. I've setup a new hobbit server on a new Solaris 10 x86 server and have seen in the last 48hours it's gone purple at least 3 times, the only client on this server is the server itself so it's not a load issue. The clients going purple are on our live monitoring solution, I have tried them in both local and nonlocal mode but with similar results every now and again they will go purple for no reason. I've added -debug to the hobbitd lines in an effort to debug the problem but as yet I havn't found anything, has anyone else come across any similar issues? Regards, Mike Rowell ________________________________________________________________________ This email has been scanned for all viruses by the MessageLabs service. ________________________________________________________________________

1 0

Alternate pagesets layout
by Craig_Whilding＠mentor.com 09 Aug '06

09 Aug '06

Hi Henrik, Having to create my first alternate pageset for a site who want to see all their stuff under more pages for a wall mounted display. Should alternate pageset pages turn up in two columns similar to a normal page with subpages? At the moment mine are all going down in what will be one huge column, is this correct? Thanks, Craig Whilding IT Systems Administrator Edward Court Altrincham Business Park Altrincham WA14 5GL Tel: + 44 (0)161 926 1643 email: craig_whilding at mentor.com www.mentor.com <file:///C:\Documents%20and%20Settings\cwhildin\Application%20Data\Micro soft\Signatures\www.mentor.com>

1 1

noconn and Enable/disable CGI
by Dominique.Frise＠unil.ch 09 Aug '06

09 Aug '06

Hi, We have hosts split across multiple pages and for service or OS grouping, same hosts can appear on several pages. Our admins are complaining that if they are viewing a page were there are hosts with the "noconn" tag, these do not appear in the Enable/disable hosts selection. It seems that the "Acknowledege Alert" CGI with the --no-pin flag has the same kind of behaviour. Is there a -hidden- configuration solution for this? Dominique UNIL - University of Lausanne

3 9

Hobbitd_client test???
by Vernon.Everett＠dpi.wa.gov.au 09 Aug '06

09 Aug '06

Hi all More problems. :-( What is causing this problem, and how can I fix it. A new test suddenly appeared called hobbitd_client and it reported red on my hobbit server line. In clientdata.log I found this. --- snip --- 2006-08-09 09:47:15 Want msg 1138, startpos 608983, fillpos 629463, endpos -1, usedbytes=20480, bufleft=7763240 2006-08-09 09:47:15 hobbitd_client: Got message 1138 @@client#1138|1155088035.931883|127.0.0.1|shire|sunos|sunos 2006-08-09 09:47:15 startpos 642855, fillpos 642855, endpos -1 2006-08-09 09:47:15 2006-08-09 09:47:17 Worker process died with exit code 134, terminating 2006-08-09 09:47:17 Want msg 1, startpos 0, fillpos 0, endpos -1, usedbytes=0, bufleft=8392703 --- snip --- Any ideas? Regards Vernon

2 1

[hobbit] Note: Hobbit 4.2 release planned for Thursday Aug. 10.
by brodie＠mcw.edu 09 Aug '06

09 Aug '06

I will try to squeeze in time to test the "duplicate data" error in my RRD data. I do not yet have the latest snapshot + all-in-1... (how to we get the snapshot and all in 1?) --------------------------------------------------------- Kent C. Brodie - brodie at phys.mcw.edu Department of Physiology Medical College of Wisconsin (414) 456-8590 -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Tuesday, August 08, 2006 4:22 PM To: hobbit at hswn.dk Subject: [hobbit] Note: Hobbit 4.2 release planned for Thursday Aug. 10. Unless some major showstopper arrives within 24 hours, the 4.2 release will be available Thursday morning around 8 AM UTC. In other words, this is your last chance to report bugs that must be fixed before the final release. The "ps" handling bugs should all be fixed now in the latest snapshot and all-in-one patch (dated 2006-08-08 21:13 UTC). A list of the bugs that have been fixed during the RC testing can be found at the beta-patches page, http://www.hswn.dk/hobbitsw/betapatches/ I would like to express my sincere admiration and thanks to the people who have taken the trouble of testing Hobbit for this release; your input and help in troubleshooting the problems has really made this release a lot better than it was 4 weeks ago. Thanks, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

2 1

[hobbit] Thank you Henrik, clientupdate, logfetch, ps parsing
by David.Gore＠VerizonBusiness.com 08 Aug '06

08 Aug '06

Henrik, Thank you for all the hard work! I can not tell you how better it is to proactive instead of reactive thanks to Hobbit. Thank you for working with me on clientupdate, logfetch and the ps parsing bugs. I am planning on taking a big rest from snapshots with the new release and will concentrate on installing it on our numerous hosts. The clientupdate feature really is nice, a very powerful tool! ~David

1 0

I found my RRD weirdness with illegal updates (2 in same time) - probable idea or fix required to resolve
by brodie＠mcw.edu 08 Aug '06

08 Aug '06

Hi everyone. Some time ago, I asked about this: 2006-08-02 12:24:50 RRD error updating /home/hobbit/data/rrd/taylor.brc.mcw.edu/disk,root.rrd from 141.106.224.178: illegal attempt to update using time 1154539490 when last update time is 1154539490 (minimum one second step) Nobody had anything useful for me, because there could be tons of causes.... However, just now, I finally figured it out. Seems that the previous sysadmin decided to create a home directory for "root" to store all his files (tarballs, etc). It was mounted as simply, "/root". This is all perfectly valid, *UNTIL* we get to rrd and disk usage. Within the realm of hobbit and rrd, the actual root file system ("/") is stored in rrd as /home/hobbit/data/rrd/taylor.brc.mcw.edu/disk,root.rrd When the client system had both "/" and "/root", this had the nifty side effect of causing hobbit'd RRD environment to become confused-- that's why I had an extra graph that had a placeholder, but nothing was being generated! That's because I had 17 mounted filesystems, which (divided by 4) is 5 graphs. But the 5th graph never showed up, and I looked carefully and figured out it was "/root" that wasn't being graphed. That's when the lightbulb went off. Two file systems "/" and "/root" were being called the same thing in terms of rrd: "root". Oooops. SO.......... While I decided to simply move the "/root" crud to somethere that made more sense... I have apparently tripped upon something that hobbit+rrd isn't prepared to deal with, which is a file system called "/root". While I think this would be rare, it's possible. Not sure how it would be handled? Maybe call the "/" file system "root~" or include some other special character in the name? --------------------------------------------------------- Kent C. Brodie - brodie at phys.mcw.edu Department of Physiology Medical College of Wisconsin (414) 456-8590

2 1