July 2007 - Xymon - Xymon mailman web

File size graph out of whack....
by rgordonjr＠gmail.com 16 Jul '07

16 Jul '07

Hello - I am attempting to keep track of the file size for /etc/aliases (and related files) on a Linux distro derived from RHEL3... The file size is 10k and is reported in the client data like this: [file:/etc/aliases] type:100000 (file) mode:644 (-rw-r--r--) linkcount:1 owner:0 (root) group:0 (root) size:10700 clock:1183753801 (2007/07/06-13:30:01) atime:1175989574 (2007/04/07-16:46:14) ctime:1183639782 (2007/07/05-05:49:42) mtime:1183639782 (2007/07/05-05:49:42) The graph though has a scale of 0 to 5.0E (whatever that is) and is showing the filesize of this file as 45956150067201.0 .... client-local.cfg has this in it: [system] log:/var/log/maillog:10240 file:/etc/aliases file:/etc/aliases.bak file:/etc/aliases.db hobbit-clients has this: HOST=system LOAD 10 20 PROC sendmail TRACK=sendmail FILE /var/log/maillog TRACK=maillog FILE /etc/aliases red size>4096 TRACK=aliases FILE /etc/aliases.bak yellow size>4096 TRACK FILE /etc/aliases.db red size>4096 TRACK PORT LOCAL=%[.:]22$ STATE=LISTEN MIN=1 MAX=-1 "TEXT=SSH listener" PORT LOCAL=%[.:]22$ STATE=ESTABLISHED MIN=0 MAX=10 TRACK=ssh "TEXT=SSH Connections" PORT LOCAL=%[.:]25$ TRACK=sendmail MIN=0 "TEXT=Sendmail Connections (IN)" PORT REMOTE=%[.:]25$ TRACK=sendmail MIN=0 "TEXT=Sendmail Connections (OUT)" Am I missing something here? -- --==[ Bob Gordon ]==--

1 1

No bbdisplay
by Steve.DiSorbo＠yale.edu 16 Jul '07

16 Jul '07

Hi all, I just installed hobbit monitor 4.2.0 on Solaris 8, hobbit starts fine, but bb.html remains blank, I tired bb2.html and bbnk.html and there is data there. Also tried cgi script find a host, but probably has nothing to do with the main display page, bb.html Any help is appreciated. Here are my log files... # more /var/log/hobbit/* :::::::::::::: /var/log/hobbit/acknowledge.log :::::::::::::: :::::::::::::: /var/log/hobbit/bb-display.log :::::::::::::: 2007-07-13 18:05:40 Whoops ! bb failed to send message - timeout 2007-07-13 18:05:40 hobbitd status-board not available :::::::::::::: /var/log/hobbit/bb-network.log :::::::::::::: 2007-07-13 18:05:40 Whoops ! bb failed to send message - timeout 2007-07-13 18:06:27 Whoops ! bb failed to send message - timeout :::::::::::::: /var/log/hobbit/bb-retest.log :::::::::::::: :::::::::::::: /var/log/hobbit/bbcombotest.log :::::::::::::: :::::::::::::: /var/log/hobbit/clientdata.log :::::::::::::: :::::::::::::: /var/log/hobbit/history.log :::::::::::::: :::::::::::::: /var/log/hobbit/hobbitclient.log :::::::::::::: 2007-07-13 18:05:47 Whoops ! bb failed to send message - timeout 2007-07-13 18:10:49 Whoops ! bb failed to send message - timeout :::::::::::::: /var/log/hobbit/hobbitd.log :::::::::::::: 2007-07-13 18:04:48 Setup complete 2007-07-13 18:34:13 Setup complete :::::::::::::: /var/log/hobbit/hobbitd.pid :::::::::::::: 26021 :::::::::::::: /var/log/hobbit/hobbitlaunch.log :::::::::::::: 2007-07-13 18:04:48 hobbitlaunch starting 2007-07-13 18:04:48 Loading tasklist configuration from /usr/local/hobbit/server/etc/hobbitlaunch.cfg 2007-07-13 18:04:48 Loading hostnames 2007-07-13 18:04:48 Loading saved state 2007-07-13 18:04:48 Cannot access checkpoint file /usr/local/hobbit/server/tmp/hobbitd.chk for restore 2007-07-13 18:04:48 Setting up network listener on 0.0.0.0:1984 2007-07-13 18:04:48 Setting up signal handlers 2007-07-13 18:04:48 Setting up hobbitd channels 2007-07-13 18:04:48 Setting up logfiles 2007-07-13 18:34:13 hobbitlaunch starting 2007-07-13 18:34:13 Loading tasklist configuration from /usr/local/hobbit/server/etc/hobbitlaunch.cfg 2007-07-13 18:34:13 Loading hostnames 2007-07-13 18:34:13 Loading saved state 2007-07-13 18:34:13 Setting up network listener on 0.0.0.0:1984 2007-07-13 18:34:13 Setting up signal handlers 2007-07-13 18:34:13 Setting up hobbitd channels 2007-07-13 18:34:13 Setting up logfiles :::::::::::::: /var/log/hobbit/hobbitlaunch.pid :::::::::::::: 26020 :::::::::::::: /var/log/hobbit/hostdata.log :::::::::::::: :::::::::::::: /var/log/hobbit/notifications.log :::::::::::::: :::::::::::::: /var/log/hobbit/page.log :::::::::::::: 2007-07-13 18:35:05 Whoops ! bb failed to send message - timeout :::::::::::::: /var/log/hobbit/rrd-data.log :::::::::::::: :::::::::::::: /var/log/hobbit/rrd-status.log :::::::::::::: -- Steve DiSorbo System Programmer Yale University ITS, AM&T Library Systems Voice (203) 432-6694 Fax (203) 436-4067 steve.disorbo at yale.edu http://www.library.yale.edu/

3 11

Hobbitd fails to start on OS X
by vadud3＠gmail.com 15 Jul '07

15 Jul '07

hobbitlaunch.log on OS X shows this. How do I fix the `shm' issue? 2007-07-15 11:27:22 hobbitlaunch starting 2007-07-15 11:27:22 Loading tasklist configuration from /Users/hobbit/server/etc/hobbitlaunch.cfg 2007-07-15 11:27:22 Loading hostnames 2007-07-15 11:27:22 Loading saved state 2007-07-15 11:27:22 Cannot access checkpoint file /Users/hobbit/server/tmp/hobbitd.chk for restore 2007-07-15 11:27:22 Setting up network listener on 0.0.0.0:1984 2007-07-15 11:27:22 Setting up signal handlers 2007-07-15 11:27:22 Setting up hobbitd channels 2007-07-15 11:27:22 Could not attach shm Too many open files 2007-07-15 11:27:22 Cannot setup user channel 2007-07-15 11:27:22 Task hobbitd terminated, status 1 http://[url/hobbit/ shows dir. ps -auxwww | grep hobbitd shows no hobbitd running -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu

2 1

RHEL3 Client runs for 30minutes then stops sending data
by aaron.stranberg＠gmail.com 14 Jul '07

14 Jul '07

All, I have a rhel3 system running the hobbit-4.2.0 client, it was installed from source. After invoking it with ./runhobbit.sh as the hobbit user the client runs long enough to return good data for the preset check (cpu/mem/etc...) to my hobbit host. After around 30minutes all agent based tests turn purple. I see nothing of interest in the log files: 2007-07-13 17:19:10 hobbitlaunch starting 2007-07-13 17:19:10 Loading tasklist configuration from /usr/lib/hobbit/client/etc/clientlaunch.cfg ps -ef |grep hobbit shows: root 29225 18127 0 17:18 pts/3 00:00:00 su - hobbit hobbit 29226 29225 0 17:18 pts/3 00:00:00 -bash hobbit 29302 1 0 17:19 ? 00:00:00 /usr/lib/hobbit/client/bin/hobbitlaunch --config=/usr/lib/hobbit/client/etc/clientlaunch.cfg --log=/usr/lib/hobbit/client/logs/clientlaunch.log --pidfile=/usr/lib/hobbit/client/logs/clientlaunch.cpdbtest.pid hobbit 29523 1 0 17:24 ? 00:00:00 sh -c vmstat 300 2 1>/usr/lib/hobbit/client/tmp/hobbit_vmstat.cpdbtest.29506 2>&1; mv /usr/lib/hobbit/client/tmp/hobbit_vmstat.cpdbtest.29506 /usr/lib/hobbit/client/tmp/hobbit_vmstat.cpdbtest hobbit 29525 29523 0 17:24 ? 00:00:00 vmstat 300 2 root 29540 14345 0 17:24 pts/5 00:00:00 grep hobbit I tried applying the all in one patch and recompiling with make then make install to no avail. Any suggetions on how to continue trouble shooting would be helpful. Also tcpdump on the host sytem and the client system show only a short burst of information being sent over port 1984 when the clients first starts then nothing after that. Thanks -Aaron

2 1

Moving from BB to Hobbit
by jwprkr＠gmail.com 13 Jul '07

13 Jul '07

Hi Hobbits, I have read the BB to Hobbit migration guide, and I have a question that I don't think was covered by the document directly. I have both a BB and test Hobbit servers up and running. I have a few of the monitored clients sending updates to both servers as a test. Now that I am convinced that Hobbit is ready to be put in production, I would like to switch the two, i.e., just switch the hostnames and IP's. and put the hobbit server in the BB server's place. This would save us from having to change the BBDISPLAY host on all the monitored clients. All of the Hobbit server software was compiled and built on the test system with the test name and IP. Can I just change the IP and hostname on the test machine to the production server and bring the BB server down and be OK, or will I have to re-compile everything once I switch them? Thanks in advance, Jeff -- Jeff Parker jwprkr at gmail dot com

6 7

RHEL3 client runs for 30 minute then stops sending data
by aaron.stranberg＠gmail.com 13 Jul '07

13 Jul '07

All,

1 0

No RRD file created from status messages generated by devmon
by vadud3＠gmail.com 13 Jul '07

13 Jul '07

On 7/13/07, <....> wrote: > > > Hello > > I explain in the how to how to graph from devmon. > http://www.hobbitmon.com/hobbiton/2007/07/msg00135.html > > In your case you are probably missing information in your template. > > Add this to the end of the messages file in the cup directory of your > template. > >  Excellent. That did it! Then you will get rrd file. > > > -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu

1 0

No RRD file created from status messages generated by devmon
by vadud3＠gmail.com 13 Jul '07

13 Jul '07

I am still looking for help with RRD. Can someone explain why I do not see any RRD files from this output? The only rrd files I see are the following (hobbit)@hobbit:~/data/rrd/svl-lead-01.inet.qwest.net$ ls tcp.conn.rrd tcp.http.https:,,svl-lead-01.inet.qwest.net,.rrd tcp.ssh.rrd On 6/25/07, Asif Iqbal <vadud3 at gmail.com> wrote: > > Hi All > > I have the following entry in bb-hosts file > > 1.2.3.4 host.domain.net # DEVMON > > It creates nice status message but no rdd and no graph gets create. > > bbcmd hobbitd_channel --channel=status hobbitd_capture --tests=cpu for > this host shows the following: > > ## @@status#409517 1182798409.418893 127.0.0.1 host.domain.net cpu > 1182800209 green green 1182795144 0 0 0 > status host,domain,net.cpu green Mon Jun 25 15:06:48 2007 > > &green Up for 17 days, 22:03:01 > &green Avg number of processes in last 1 min: 233 > &green Avg number of processes in last 5 mins: 254 > &green Avg number of processes in last 15 mins: 254 > > System Description: Peakflow SP 3.6.1 (build 7DQJ) Model: SE7501WV2S > Serial number: ADLP5476456 > > CPULoadAvg : 254 > > > Devmon version 0.3.0-beta2 running on hobbit > > > Should I not see a ~hobbit/data/rrd/host.domain.net/la.rrd file? > > -- > Asif Iqbal > PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu > > > -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu

1 0

How to if_* graph from Devmon to Hobbit
by xbgmsharp＠gmail.com 13 Jul '07

13 Jul '07

How to if_* graph from Devmon to Hobbit This mail describe how to graph all if_* data from Devmon to Hobbit. I am currently running hobbit 4.2.0/all-in-one patch and devmon 0.3-beta2 1) First you need a curently working hobbit and devmon. 2) Then make sure, than the file devmon/templates/modele/if_load/message TABLE:rrd() This will show data in a htmltable and add rrd information in the status page. In order to check, you should see RRD data information in the status page of the service corresponding to your devmon modele when showing source html. 3) Now make parse the data to create a rrd file (one par interfaces), this is done via extra-rrd.pl. In etc/hobbitlaunch.cfg [rrdstatus] ENVFILE /data/hobbit/server/etc/hobbitserver.cfg NEEDS hobbitd CMD hobbitd_channel --channel=status \ --log=$BBSERVERLOGS/rrd-status.log hobbitd_rrd \ --rrddir=$BBVAR/rrd \ --extra-script=$BBHOME/ext/extra-rrd.pl \ --extra-tests=if_load In ext/extra-rrd.pl if ( $TESTNAME eq "if_load" ) { # Analyze the message we got foreach $line (@input) { if ($line =~ /(^[A-Z].*) (\d+):(\d+)$/ ) { # The RRD dataset definitions print "DS:in:DERIVE:600:0:U\n"; print "DS:out:DERIVE:600:0:U\n"; # The filename print "if_load_$1.rrd\n"; # The data print "$2:$3\n"; } } } 4) Modify etc/hobbitgraph.cfg [if_load] FNPATTERN if_load(.*).rrd TITLE Network Traffic YAXIS Bits/second DEF:in at RRDIDX@=@RRDFN@:in:AVERAGE CDEF:inbytes at RRDIDX@=in at RRDIDX@,8,* DEF:out at RRDIDX@=@RRDFN@:out:AVERAGE CDEF:outbytes at RRDIDX@=out at RRDIDX@,8,* LINE1:inbytes at RRDIDX@#@COLOR@:@RRDPARAM@ In GPRINT:inbytes at RRDIDX@:LAST: %8.2lf %s (cur) GPRINT:inbytes at RRDIDX@:MAX: %8.2lf %s (max) GPRINT:inbytes at RRDIDX@:MIN: %8.2lf %s (min) GPRINT:inbytes at RRDIDX@:AVERAGE: %8.2lf %s (avg)\n LINE1:outbytes at RRDIDX@#@COLOR@:@RRDPARAM@ Out GPRINT:outbytes at RRDIDX@:LAST: %8.2lf %s (cur) GPRINT:outbytes at RRDIDX@:MAX: %8.2lf %s (max) GPRINT:outbytes at RRDIDX@:MIN: %8.2lf %s (avg) GPRINT:outbytes at RRDIDX@:AVERAGE: %8.2lf %s (avg)\n 4) In etc/hobbitserver.cfg Modify the following. See help/howtograph.html TEST2RRD="cpu=la,disk,...,if_load" GRAPHS="la,disk,if_load::1" The ::1 set the maxcount in order to make one graph form each rrd file create. 5) In etc/hobbitcgi.cfg Modify the following. See hobbitsvc.cgi man CGI_SVC_OPTS="--env=/data/hobbit/server/etc/hobbitserver.cfg --no-svcid --history=top --multi-grahs=if_load,if_err..." 6) The hardest part. The hobbit parsing doesn't remove html table table so you will see many graph link. In order to fix than, you need to modify the hobbit source with the patch: lib/htmllog.c $cd lib/ $patch -p0 < htmllog.path Here is the path, this only set a new counter for all if_* services. This is allready working for the trends colunms because, it is checking all rrd file in the directory. Henrik, do you have any tips to get the nice counter without editing the source code of htmllog.c file ? That's all folks.

1 0

Debugging a client
by trent.melcher＠sitel.com 12 Jul '07

12 Jul '07

Ok, our firewall team just opened up port 1984 so a client could connect to our hobbit server. Well I can telnet to the server from the client on port 1984, but Im still getting timeout messages in the hobbitclient.log, below is a bb connect with debug on, what am I missing? [root at maila logs]# /home/hobbit/client/bin/bb --debug XXXXXXXXXXX "status xxxxx,xxxxx,net.test green `date` This is a test" 2007-07-12 08:26:41 Transport setup is: 2007-07-12 08:26:41 bbdportnumber = 1984 2007-07-12 08:26:41 bbdispproxyhost = NONE 2007-07-12 08:26:41 bbdispproxyport = 0 2007-07-12 08:26:41 Recipient listed as 'xxxxx.xxxxx.net' 2007-07-12 08:26:41 Standard BB protocol on port 1984 2007-07-12 08:26:41 Will connect to address xxxxx.xxxxx.net port 1984 2007-07-12 08:26:41 Connect status is 0 2007-07-12 08:26:41 Sent 77 bytes 2007-07-12 08:26:41 Closing connection [root at maila logs]# telnet xxxxx.xxxxx.net 1984 Trying XX.XX.XX.XX... Connected to xxxxx.xxxxx.net (XX.XX.XX.XX). Escape character is '^]'. ^] telnet> Thanks Trent

2 7