May 2010 - Xymon - Xymon mailman web

hobbitd_client on web page is purple
by kkelly＠lifetouch.com 20 May '10

20 May '10

Hobbitd_client is showing purple status on the web page, I would like to get rid of it. I have tried: Stop/start of xymon server with no luck. bb hostname "drop hostname hobbit_alert" no luck there either! Does anyone know how to clear this status? Thanks ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email _____________________________________________________________________

2 2

AUTO: Chris Ahrens is out of the office (returning 06/01/2010)
by chris.ahrens＠macys.com 20 May '10

20 May '10

I am out of the office until 06/01/2010. Note: This is an automated response to your message "[hobbit] Memstat for Solaris (10)" sent on 5/20/10 2:17:56. This is the only notification you will receive while this person is away.

1 0

Unknown token error question
by rmcgraw＠purdue.edu 17 May '10

17 May '10

I am getting the following lines in clientdata.log but not sure what file it is referencing that has the Unknown token. 2010-05-17 17:37:07 Unknown token 'client' ignored at line 464 2010-05-17 17:37:07 Unknown token 'server' ignored at line 465 What file is it referencing? Thanks Robert _____________________________________________________________________ Robert P. McGraw, Jr. Manager, Computer System EMAIL: rmcgraw at purdue.edu Purdue University ROOM: MATH-807 Department of Mathematics PHONE: (765) 494-6055 150 N. University Street West Lafayette, IN 47907-2067

2 1

Question about MySQL Replication Script
by deger＠cobercafe.net 17 May '10

17 May '10

I tried to get this script working but it never seems to send data back to the hobbit server. I ran the script in debug mode, and it seems to sending the correct information, but not sure why it not showing up on the hobbit. [root at cs101 ext]# ./mysql-replication.pl /home/hobbit/client/bin/bb 10.1.5.18 "status 10,1,5,39.replication green Mon May 17 20:14:22 2010 Mirroring OK Node: node master T1 at mysql-bin.000302:949148896 (10.1.5.39) slave RUNNING (10.1.5.44) slave ON TIME at mysql-bin.000302:949164080 (10.1.5.44) master T2 at mysql-bin.000302:949179136 (10.1.5.39) " #!/usr/bin/perl -w # # mysql replication monitoring # generic attempt use strict; #use Data::Dumper; my $DEBUG = 0; $ENV{BBPROG} = "bb-mysql-replication"; my $TESTNAME = "mysqlrep"; my $BBHOME = $ENV{BBHOME}; my $BB = $ENV{BB}; # full path to the bin/bb util my $BBDISP = $ENV{BBDISP}; # IP of the BBDISPLAY server my $BBVAR = $ENV{BBVAR}; my $MACHINE = $ENV{MACHINE}; # hostname, fqdn my $COLOR = "clear"; # global color for the test my $MSG = ""; # body of the message my $HEAD = ""; # first line of the message (has to be short, optional) my $DATA = ""; # data for NCV records (hobbit only) if ($DEBUG == 1) { $BBHOME = "/tmp" unless $BBHOME; $BBDISP = "127.0.0.1" unless $BBDISP; $BBVAR = "/tmp" unless $BBVAR; $MACHINE = "host.priv.com" unless $MACHINE; } # The Fine Manual : sub clear; # sets status color to clear sub green; # ~ to green sub yellow; # ~ to yellow sub red; # ~ to red sub setcolor; # safely sets status color # setcolor("yellow") sub head; # sets the first status line # head("foo OK") sub msg; # adds text to the body of the status message # msg("foo" [, "bar", ...]) sub data; # adds NCV-formatted data to the status message # data(ds, value) sub sendreport; # sends the report to the BB server # sendreport() sub resetreport; # wipes the report (head, body, data, color) # resetreport() # You can always use $MSG, $HEAD, $DATA and $COLOR directly. ######## # Put your code here ######## # What you need : # on the remote masters and slaves, create: # for mysql 4.x : grant replication client on *.* to 'user'@'monitorhost' identified by 'password'; # grant select on test.* to 'user'@'monitorhost' identified by 'password'; # for mysql 3.x : grant process on *.* to 'user'@'monitorhost' identified by 'password'; # grant select on test.* to 'user'@'monitorhost' identified by 'password'; # use DBI; my $ERR; my $legalERR = "[^a-zA-Z0-9.:,_ -]"; my ($status, $maitreSql, @slaves, $esclaveSql, $login, $pass, %auth, $node); my ($positionMaitre_t1, $positionMaitre_t2, $positionEsclave, $slaveRunning, $error); my $conffile="/usr2/hobbitlocal/etc/bb-mysql-replication.cfg"; my $OKOFFSET = 100; readConfig(); foreach $node ( keys %auth ) { msg "Node: $node\n"; green; # let's be optimit ;) $maitreSql = $auth{$node}{master}; @slaves = keys %{ $auth{$node}{slaves} }; $login = $auth{$node}{user}; $pass = $auth{$node}{pass}; $MACHINE = $maitreSql; $positionMaitre_t1 = getSqlMasterStatus($maitreSql,$login,$pass,"test"); if (not defined $positionMaitre_t1) { $ERR =~ s/$legalERR/ /go; msg(sprintf("master ERROR (%s) &red %s", $maitreSql, $ERR)); head "mirroring FAILURE"; sendreport; resetreport; next; } sleep 2; foreach $esclaveSql (@slaves) { ($positionEsclave, $slaveRunning,$error) = getSqlSlaveStatus($esclaveSql,$login,$pass,"test"); next if not defined $positionEsclave; $auth{$node}{slaves}{$esclaveSql}{pos} = $positionEsclave; $auth{$node}{slaves}{$esclaveSql}{running} = $slaveRunning; } sleep 2; # on reprend la position sur le maître $positionMaitre_t2 = getSqlMasterStatus($maitreSql,$login,$pass,"test"); if (not defined $positionMaitre_t2) { head "mirroring FAILURE"; $ERR =~ s/$legalERR/ /go; msg(sprintf("master ERROR (%s) &red %s", $maitreSql, $ERR)); sendreport; resetreport; next; } msg(sprintf("master T1 at % 40s (%s)", $positionMaitre_t1, $maitreSql)); foreach $esclaveSql (@slaves) { $positionEsclave = $auth{$node}{slaves}{$esclaveSql}{pos}; $slaveRunning = $auth{$node}{slaves}{$esclaveSql}{running}; if (not defined $positionEsclave) { $ERR =~ s/$legalERR/ /go; msg "slave ERROR ($esclaveSql) &red $ERR"; next; } elsif ($slaveRunning eq "Yes") { msg "slave RUNNING ($esclaveSql)"; if ( ($positionMaitre_t1 le $positionEsclave) and ($positionEsclave le $positionMaitre_t2) ) { msg(sprintf("slave ON TIME at % 40s (%s)", $positionEsclave, $esclaveSql)); } else { my($masterfile, $masteroffset, $slavefile, $slaveoffset); ($masterfile, $masteroffset) = split(/:/, $positionMaitre_t1, 2); ($slavefile, $slaveoffset) = split(/:/, $positionEsclave, 2); msg(sprintf("slave LATE at % 40s (%s) &yellow", $positionEsclave, $esclaveSql)); if($masterfile eq $slavefile) { if ($masteroffset - $slaveoffset < $OKOFFSET) { green; } else { yellow; } } else { $masterfile =~ s/.+\.(\d+)$/$1/; $slavefile =~ s/.+\.(\d+)$/$1/; if($masterfile - $slavefile != 1) { yellow; } elsif($masteroffset < $OKOFFSET) { green; } else { yellow; } } } } else { msg "slave STOPPED ($esclaveSql) &red"; red; } if($error ne "") { red; $error =~ s/$legalERR/ /go; msg "slave ERROR $error"; } } msg(sprintf("master T2 at % 40s (%s)", $positionMaitre_t2, $maitreSql)); if ($COLOR eq "green") {head "Mirroring OK"} else {head "Mirroring FAILURE"} sendreport; resetreport; } exit 0; ############### ############### ######################## # conffile has format : # node;masterhost;slavehost1[,slavehost2,...];user;pass # (so it works in multiple slave setups) # example: # authcluster;auth-sql01.priv.com;auth-sql02.priv.com;monitor;coUIc # radius;rad-sql01.priv.com;rad-repsql01.priv.com,rad-backup01.priv.com;monitor;coUIc # add comment lines with '#' sub readConfig { my ($node, $master, $slaves, $user, $pass, @conf, $line); if (! open(CONF, "$conffile")) { head("monitoring error"); msg("Erreur à l'ouverture de $conffile : $!"); red; sendreport; exit 1; } @conf = <CONF>; close(CONF); foreach $line (@conf) { if (substr($line, 0, 1) eq "#") { next } elsif (substr($line, 0, 1) eq "\n") { next } else { ($node, $master, $slaves, $user, $pass) = split(/\s*;\s*/, $line, 5); if ($pass) { chomp $pass; $auth{$node}{master} = $master; foreach my $slave (split(",", $slaves)) { $auth{$node}{slaves}{$slave}{pos} = undef; $auth{$node}{slaves}{$slave}{running} = "not tested"; } $auth{$node}{user} = $user; $auth{$node}{pass} = $pass; } else { msg("&red invalid config : $line"); red; } } } #print Dumper \%auth; } # Retourne les infos relatives au sql maitre # <- (Position); sub getSqlMasterStatus { my ($host,$user,$pass,$base) = @_; my $port; ($host, $port) = split(/:/, $host, 2); if(not defined $port) { $port = 3306; } my $dbLink = DBI->connect("DBI:mysql:$base:$host:$port:mysql_connect_timeout=10","$user","$pass"); if (! $dbLink) { $ERR = $DBI::errstr; red; return (undef); } my $query = $dbLink->prepare("SHOW MASTER STATUS"); if (! $query->execute()) { $ERR = $DBI::errstr; red; return (undef); } my @results = $query->fetchrow_array; return "$results[0]:$results[1]"; } # Retourne les infos relatives au sql slave # <- (Exec_Master_Log_Pos, Slave_IO_Running&&Slave_SQL_Running) sub getSqlSlaveStatus { my ($host,$user,$pass,$base) = @_; my ($port, $slaverunning); ($host, $port) = split(/:/, $host, 2); if(not defined $port) { $port = 3306; } my $dbLink = DBI->connect("DBI:mysql:$base:$host:$port:mysql_connect_timeout=10","$user","$pass"); if (!$dbLink) { red; $ERR = $DBI::errstr; return (undef); } my $query = $dbLink->prepare("SHOW SLAVE STATUS"); $query->execute(); if (! $query->execute()) { $ERR = $DBI::errstr; red; return (undef); } my @results = $query->fetchrow_array; ## sloppy but works :-/ ## mysql 3.23 if ((scalar @results) == 12) { return ("$results[4]:$results[5]",$results[6], $results[10]); } ## mysql 4.0 elsif ((scalar @results) == 18) { if($results[9] eq "Yes" and $results[10] eq "Yes") { $slaverunning = "Yes" } else { $slaverunning = "No"; } return ("$results[8]:$results[16]", $slaverunning, $results[14]); } ## mysql 4.1 elsif ((scalar @results) == 33) { if($results[10] eq "Yes" and $results[11] eq "Yes") { $slaverunning = "Yes" } else { $slaverunning = "No" } return ("$results[9]:$results[21]", $slaverunning, $results[19]); } ## mysql x.q ? else { red; msg("Unsupported mysql version"); return (undef, "unknown"); } } ################### ################### #### # send the report #### sub sendreport { $MACHINE =~ s/\./,/g; my $date = localtime; my $cmd = "$BB $BBDISP \"status $MACHINE.$TESTNAME $COLOR $date $HEAD\n$DATA\n$MSG\""; if($DEBUG == 1) { print "$cmd\n"; } else { system($cmd); } } sub resetreport { $MSG = $DATA = $HEAD = ''; $COLOR = 'clear'; } # sets the global color of the test # prevents downgrading severity # clear == green < yellow < red sub setcolor { my $newcolor = shift; if ($newcolor eq "red") { $COLOR = "red"; } elsif ($COLOR eq "green" or $COLOR eq "clear") { $COLOR = "$newcolor"; } return $COLOR; } sub clear { setcolor 'clear' } sub green { setcolor 'green' } sub yellow { setcolor 'yellow' } sub red { setcolor 'red' } sub data { my ($n, $v) = @_; $DATA .= "$n: $v\n"; } sub head { $HEAD = "@_"; } sub msg { $MSG .= join("\n", @_) . "\n"; }

1 0

how to monitor switches and increase the polling time
by pankaj.dorlikar＠gmail.com 17 May '10

17 May '10

Hi, since we are monitoring ober 300 servers at a time in a custer, we want to know and analyser the network traffic in ethernet and infiniband switch .how we can monitor it using xymon system monitoring.os is rhel 5.2 and xymon version is 4.2.2 Practical problem happening is that some of the nodes reboots but xymon doesnt show it. it may be because that happens in betwwen the polling period of the server. how to overcome this issue?same thing happens for disk and ncpu and external tests also that it does not identify and show these problems as it must be occuring in betwwn polling period .How to make it fast as ganglia -- Pankaj V. Dorlikar

3 5

Additional test not showing
by David.Hurst＠t-systems.co.za 17 May '10

17 May '10

Hi All, I am (very) new to Hobbit, and have searched for an answer to this one - but the answers I seem to find include writing a script to obtain certain information and post it to Hobbit. Basically, this pertains to the tests being done on my mainframe (z/OS). I have the z/OS monitor running there (works great) & have changed some code to pass some database information back to my hobbit server (only very simple information at the moment - until I get this working), so it looks like this in the "client data" on the server : client mf.zos zos [osversion] z/OS 01.10.00 HBB7750 JES2 z/OS1.10 [date] Thu May 13 06:59:10 CST 2010 [uptime] Uptime: 11 Days, 11 Hours, 6 Minutes [cpu] CPU Utilization 10% [paging] ............ [databases] ADADB029 Database 029 The [databases] at the bottom is a new test that I am performing, but for the life of me cannot get it to display in a new column on my hobbit web page. How do I tell Hobbit that there is a new "section" in the data which is passed from my mainframe that must be displayed? Or do I need to write a script to strip the information off and post it into hobbit that way? To write a script to strip the info off the mainframe data just doesn't quite make sense to me ... I am running Xymon4.3.0-0.beta2. Regards Dave Disclaimer: This message and/or attachment(s) may contain privileged or confidential information. If you are not the intended recipient you may not disclose or distribute any of the information contained within this message. In such case you must destroy this message and inform the sender of the error. T-Systems does not accept liability for any errors, omissions, information and viruses contained in the transmission of this message. Any opinions, conclusions and other information contained within this message not related to T-Systems' official business is deemed to be that of the individual only and is not endorsed by T-Systems. This message and/or attachment(s) may contain privileged or confidential information. If you are not the intended recipient you may not disclose or distribute any of the information contained within this message. In such case you must destroy this message and inform the sender of the error. T-Systems does not accept liability for any errors, omissions, information and viruses contained in the transmission of this message. Any opinions, conclusions and other information contained within this message not related to T-Systems' official business is deemed to be that of the individual only and is not endorsed by T-Systems. T-Systems - Business Flexibility

4 3

Network test performed on hostname instead of IP
by meepmeep＠fantasya.org 12 May '10

12 May '10

Hi I use xymon to monitor a couple of irc server from my network. The bb-host file looks like this : 1.1.1.1 irc1.domain.tld # telnet:6667 2.2.2.2 irc2.domain.tld # telnet:6667 3.3.3.3 irc2.domain.tld # telnet:6667 4.4.4.4 hub1.domain.tld # telnet:6667 I works great until today : 1/ irc2.domain.tld goes down => xymon send me an alert => Monitoring works great ! 2/ I change the DNS of irc2.domain.tld to "CNAME irc.domain.tld" (round robin on other ircX.domain.tld) to redirect user. 3/ 5 minutes after the propagation of the new dns ... xymon says that irc2.domain.tld is back online (which is right for the dns, but not for the IP address ...) I check the "conn" page of "irc2.domain.tld", and it gives an ip from the round robin of irc.domain.tld ==> So, before making the test, xymon resolves the hostname instead of directly using the IP address.). For me, the hostname is only here to show something "digest" on the web page, I would like to run every test on the ip (including conn and telnet on my case).... Any idea on a way to change this ? -- Nicolas G. / meepmeep [EuropNet.org Admin]

1 1

Question about Server Status Forwarding
by captkras＠gmail.com 12 May '10

12 May '10

We are starting to deploy multiple "collector" servers. What we want is to have Hobbit Server A collect from it's local group, Server B to collect from its local group, and Server C to collect for its local group. (By local group, I mean IP address and physical location space). Then we want to have Server A and Server C send the status messages that they have collected to Server B. I THOUGHT all that needed to be done was to put in the following: [snip from hobbitserver.cfg on Server A] BBDISP="0.0.0.0" # IP of a single hobbit/bbd server BBDISPLAYS="<Server A IP>,<Server B IP>" # IP of multiple hobbit/bbd servers. If used, BBDISP must be 0.0.0.0 However, it appears that all the entries that are being forwarded are coming in as ghosts. Is there something else I need to do on ServerB? Or on the client side? Should I be using BB-proxy? There isn't any firewall or proxying between the groups. Dave -- David S. Morgan CISSP, CCNP aka: captkras at gmail.com "When the Winds of Change Blow Hard Enough, the Most trivial of things can turn into deadly projectiles"

4 10

Custom graphs in Xymon
by camis＠mweb.co.za 11 May '10

11 May '10

Hi.. I'm having an issue with creating a system graph in Xymon. The script outputs the following on the page: ----------------------------- Fri May 7 16:42:19 SAST 2010 edge01 : 5 edge02 : 7 edge03 : 2 edge04 : 6 hobbitgraph ncv:fmsconns Zoom graph ----------------------------- Its a script to monitor concurrent connections so its value goes up and down. In hobbitserver.cfg, i have the following: TEST2RRD="cpu=la,cpu2=la[......],fmsconns=ncv" GRAPHS="la,disk[...........],fmsconns" NCV_fmsconns="edge01:GAUGE,edge02:GAUGE" in hobbitgraph.cfg [fmsconns] TITLE Fmsconns info YAXIS Connections DEF:pct=fmsconns.rrd:util:AVERAGE LINE2:pct#00CC00:NParts GPRINT:pct:LAST: \: %3.1lf (cur) GPRINT:pct:MAX: \: %3.1lf (max) GPRINT:pct:MIN: \: %3.1lf (min) GPRINT:pct:AVERAGE: \: %3.1lf (avg)\n On the I used the howto @ http://www.hswn.dk/hobbit/help/howtograph.html as a guideline but its clear i'm lacking some understanding. I'm guessing the contents in hobbitgraph.cfg is off as i'm doing an rrdtool dump and the output looks ok? <lastupdate> 1273230835 </lastupdate>  <ds> <name> jhbedge01optinetnet </name> <type> DERIVE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max>  <last_ds> 4 </last_ds> <value> 0.0000000000e+00 </value> <unknown_sec> 0 </unknown_sec> </ds> <ds> <name> jhbedge02optinetnet </name> <type> DERIVE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max>  <last_ds> 3 </last_ds> <value> 7.9207920792e-02 </value> <unknown_sec> 0 </unknown_sec> </ds> Anyone have any ideas/tips? Regards, Cami

6 11

status file to column issue
by pnixon＠gmail.com 11 May '10

11 May '10

Hey all, I have a pull environment setup and I've configured the script to append to the status file and read into xymon appropriately. However, it's not create the column for the test and displaying the results. Any suggestions on how to proceed with this? client cw-moremagic2.linux [date] Mon May 10 18:38:00 BST 2010 [uname] Linux moremagic2 2.6.9-78.0.5.ELsmp i686 [osversion] Red Hat Enterprise Linux ES release 4 (Nahant Update 7) [uptime] 18:38:00 up 564 days, 16:41, 1 user, load average: 0.02, 0.09, 0.08 .... [vmstat] procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 119976 42496 14624 3721812 0 0 2 5 0 0 2 1 97 0 0 0 119972 56384 15440 3707748 0 0 1 65 1026 186 1 0 99 0 [tablespace] INDEX1 : 13.2 SYSAUX : 63.9 SYSTEM : 65.4 TABLES1 : 37.5 UNDOTBS1 : .9 USERS : 8.8 [clock] epoch: 1273513087.077486 local: 2010-05-10 18:38:07 BST UTC: 2010-05-10 17:38:07 GMT

3 4