Custom graphs in hobbit, resend, ATTN Alan Sparks

12 Oct 2007

      (This is a resend since the formatting was crapped out in the previous mail)
Hi.
The problem with parsing is to know what you want to parse.
The first step is to send your information to hobbit and see it displayed.
Second step is to configure hobbitlaunch.cfg file to send the data from the column to your script.
FROM /etc/hobbit/hobbitlaunch.cfg
[rrdstatus]
ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg
NEEDS hobbitd
CMD hobbitd_channel --channel=status --log=$BBSERVERLOGS/rrd-status.log hobbitd_rrd --extra-tests=hitastig --extra-script=/usr/lib/hobbit/server/ext/rrd_digitemp.pl --rrddir=$BBVAR/rrd The --extra-tests is the definition of what columns you want to parse. You can have as many columns as you like, comma seperated.
The --extra-script is the path to the script that you wish to run to do the parsing. You can only have one there.
Then you have to edit hobbitserver.cfg and add the test to TEST2RRD and GRAPHS variables.
Then either reboot the server or kill hobbit_channel processes (they should restart the next time they are scheduled to run)
To begin to know what you have to parse, just tell the script to write to some file (overwrite, not append .. Unless of course you want it to grow bigger and bigger). It should be very similar to what the web site shows.
Alternatively there should be a way to use the bb program to display it, but I didn't bother finding out how.
Then of course you do your parsing.
I recommend that you read about RRD graphs and how you can display different values (is it a counter, is it a variable like temperature, percentage etc etc) on the official RRD website which slips my mind ATM where it is located .. Google will probably help.
You should receive 3 args from hobbit when it runs your script, it behaves so: $scriptname $hostname $testname $messagefile The messagefile is a temporary file which you shall parse.
The final stage is to output your information to hobbit in a way it understands, printing directly to STDOUT.
The format is so, note that you don't have to have the type GAUGE, replace with what you need:
DS:<datasetname>:GAUGE:600:0:100
<Testname>.<sensorname>.rrd
<value>
DS:<datasetname>:GAUGE:600:0:100
<Testname>.<sensor2name>.rrd
<value>
Etc etc..
A tip for the wise, I had a lot of struggle with this when I was making it, and then realised that I can't have the datasetnames any different in all the files, it has to be static, that is, one name.
Then you should check if there are any RRD files being made.
They should be located in /var/lib/hobbit/rrd/<hostname>/
Then there comes the fun, to be able to show off your results in a graph. You should edit hobbitgraph.cfg file for that.
Mine looks like this:
[hitastig]
FNPATTERN hitastig.(.*).rrd
TITLE Hitastig
YAXIS Celsius
DEF:h at RRDIDX@=@RRDFN@:temperature:AVERAGE
LINE2:h at RRDIDX@#@COLOR@:@RRDPARAM@
GPRINT:h at RRDIDX@:LAST: \: %5.1lf (cur)
GPRINT:h at RRDIDX@:MAX: \: %5.1lf (max)
GPRINT:h at RRDIDX@:MIN: \: %5.1lf (min)
GPRINT:h at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n
The files it expects are hitastig.<sensorname>.rrd, don't let that extra dot fool you, it needs to be there.
See in the DEF line, the name I chose for the datasets is temperature. (I REMIND YOU AGAIN, IT NEEDS TO BE STATIC)
If you do this successfully, you should see a graph in hobbit as soon as you finish this last thing (just let the website refresh once).
For reference (and of course to future developers that will read the mailing list archive) I will attach my scripts here so you know what to expect.
dgitemp.pl -- This script reads from digitemp values, names and peak values from the MySQL database and sends them to hobbit.
#!/usr/bin/perl
This script reads digitemp values and writes them to a # temporary file which the hobbit client sends out to # the Hobbit Monitor.
This script really should be rewritten so it will contain subs for operations.
That would be much more tidier.
Written by Sigurdur Gudbrandsson
sigurdur at raforninn.is
use strict;
use DBI;
Lets declare our variables
my $db_user = "username"; # Change to whatever your username is my $db_pass = "password"; # Change to whatever your password is my $db_name = "stats"; my $table_meta = "digitemp_metadata"; my $table = "digitemp";
We need to get some variables from the environment, don't change this my $BB = $ENV{BB}; my $BBDISP = $ENV{BBDISP}; my $MACHINE = $ENV{MACHINE}; my $BBTMP = $ENV{BBTMP};
Change this to whatever you want your test to be named my $testname = "hitastig";
Don't change this or your test will never be green my $color = "green";
Set this to other than 0 if you want to debug my $debug = 0;
Lets connect to the database
my $dbh = DBI->connect("DBI:mysql:$db_name","$db_user","$db_pass")
or die "I cannot connect to dbi:mysql:$db_name as $db_user - $DBI::errstr\n";
Lets get the sensor ID's, names and min/max values
my $sql = "SELECT SerialNumber,name,description,min,max FROM $db_name.$table_meta";
my $sth = $dbh->prepare($sql) or die "Can't execute statement $sql because: $DBI::errstr"; $sth->execute();
my (@sensors) = ();
while (my @ary = $sth->fetchrow_array()) {
if ($ary[3] == "NULL") {
$ary[3] = 0.000;
}
if ($ary[4] == "NULL") {
$ary[4] = 99.999;
}
push(@sensors, [@ary]);  # [@ary] is a reference } $sth->finish();
Now we have all the sensors in a two dimensional array, @sensors # Also, if there is a NULL value in min or max, it has been substituted with a value
Now we will get the temperature of each sensor from the database # and check if the temp is too high or too low.
my $count = 0;
while (@{sensors[$count]}) {
print "WHILE: entering first while loop\n" if ($debug);
my $sql = "SELECT Fahrenheit FROM $db_name.$table WHERE SerialNumber='$sensors[$count][0]' ORDER BY dtKey DESC LIMIT 0,1";
print "SQL: term is \" $sql \" \n" if ($debug);
my $sth = $dbh->prepare($sql) or die "Can't execute statement $sql because: $DBI::errstr";
$sth->execute();
$sensors[$count][5] = $sth->fetchrow_array();
if ($sensors[$count][5] > $sensors[$count][4]){
print "IF: status for $sensors[$count][1] is red\n" if ($debug);
$sensors[$count][6] = "&red";
$color = "red";
}
elsif ($sensors[$count][5] < $sensors[$count][3]){
print "IF: status for $sensors[$count][1] is yellow\n" if ($debug);
$sensors[$count][6] = "&yellow";
if ($color != "red"){
$color = "yellow";
}
}
else {
print "IF: status for $sensors[$count][1] is green\n" if ($debug);
$sensors[$count][6] = "&green";
}
$sth->finish();
$count++;
}
We can close the database now
$dbh->disconnect();
Next is to get our results into an array # For reference, 0=SerialNumber 1=name 2=description 3=min 4=max 5=value 6=color in our array
my $tmp = "\n";         # First line in the array
print $color if ($debug);
$count = 0;
while (@{sensors[$count]}) {
print "WHILE: entering second while loop\n" if ($debug);
print "WHILE: $sensors[$count][1] is being printed\n" if ($debug);
$tmp = $tmp."    $sensors[$count][6]    $sensors[$count][1] = $sensors[$count][5]\n";
$tmp = $tmp."            : min=$sensors[$count][3] max=$sensors[$count][4]\n";
$tmp = $tmp."            : $sensors[$count][2]\n\n";
$count++;
}
Now we decode from UTF-8 to latin1 (iso-8859-1) if there is anything in utf8 open(TEMP, ">$BBTMP/$testname.tmp.txt"); print TEMP $tmp; close(TEMP);
Now we get the converted data
$tmp = /usr/bin/iconv -f utf8 -t iso-8859-1 $BBTMP/$testname.tmp.txt;
Now finally we output the information to our beloved hobbit my $date = localtime; my $cmd = "$BB $BBDISP \"status $MACHINE.$testname $color $date \n $tmp \n \""; system($cmd);
print $cmd if ($debug);
print $tmp if ($debug);
exit;
Here is the output from this script
&green    Tölvu_kæling = 21.25 C
        : min=1.000 max=25.000
        : This sensor has not yet been described

&green    Verkstæði = 25.81 C
        : min=0.000 max=99.999
        : This sensor has not yet been described

&green    Tölvuskápur = 22.12 C
        : min=14.000 max=25.000
        : This sensor has not yet been described

&green    Tölvu_kæl_út = 46.38 C
        : min=0.000 max=99.999
        : This sensor has not yet been described
rrd_digitemp.pl -- This script parses the information from hobbit and produces the channel data to make the RRD files.
#!/usr/bin/perl
use strict;
Input parameters: Hostname, testname (column), and messagefile my $hostname=$ARGV[0]; my $testname=$ARGV[1]; my $fname=$ARGV[2];
my ($line1, @line, @buff, @key, @value, $value, @tmp);
open(IN,"$fname");
read(IN, $line1, 10000);
@line=split('\n',$line1);
@buff = grep(/ = /, at line);
close(IN);
chomp(@buff);                                   # Lets strip the newline if there is any.
for( my $i = 0; $i < scalar(@buff); $i++)       # Now we go through the whole array of lines
{
$buff[$i] = substr($buff[$i], 10);      # Trimming the beginning
$buff[$i] =~ s/ C//g;                   # Removing the end
$buff[$i] =~ s/ //g;                    # Removing all the whitespace
@tmp = split('=', $buff[$i]);
$key[$i] = makesafe($tmp[0]);
$value[$i] = $tmp[1];
print $buff[$i]."\n";                   # Printing a test line to view the results
}
open(IN,">>/var/log/hobbit/custrrd.log");
write($buff[1]);
close(IN);
The next loop will print out the information for making the RRD file.
for( my $i = 0; $i < scalar(@key); $i++ ) {
print "DS:temperature:GAUGE:600:0:100\n";      # Prints out each sensor name
print "DS:".$key[$i].":GAUGE:600:0:100\n";      # Prints out each sensor name
    print $testname.".".$key[$i].".rrd\n";          # Prints out the rrd file name
    print $value[$i]."\n";                          # Prints out the value
}
This sub produces safe output so hobbit will be able to make the files # (It seems that hobbit doesn't like special letters in file names) sub makesafe {
    my $word = shift;
    $word =~ s/ö/o/g;
    $word =~ s/á/a/g;
    $word =~ s/í/i/g;
    $word =~ s/é/e/g;
    $word =~ s/ó/o/g;
    $word =~ s/ý/y/g;
    $word =~ s/ú/u/g;
    $word =~ s/æ/ae/g;
    $word =~ s/ð/d/g;
    $word =~ s/þ/th/g;
$word =~ s/_//g;
$word = lc($word);
    return $word;
}
exit;
I hope this isn't overkill. :)
With regards,

Sigurður Guðbrandsson
Raförninn ehf.
Suðurhlíð 35
105 Reykjavik | Iceland
sigurdur at raforninn.is | www.raforninn.is
Office: +(354) 552 2070
Mobile: +(354) 867 3573

-----Original Message-----
From: Alan Sparks [mailto:asparks at doublesparks.net]
Sent: 10. október 2007 21:06
To: Sigurður Guðbrandsson
Subject: Re: [hobbit] Custom graphs with multiple data sources
Since I am in the same boat with my server (downtime not an option), I am interested in your approach.  I already have custom scripts for this, but am interested in what you do differently as far as parsing.  Are you parsing the incoming messages and re-sending them to Hobbit?  What is the "proper format?"
Thanks for your reply.
-Alan
wrote:
...
Hi.
There is a fourth option, which I took as updating to snapshot caused my hobbit to break for some reason, and seeing that it is a system I must have up 99% of the day, I just backported.
Well, that was not the option anyways :)
My suggestion is that you make a custom script to parse your information and create the RRD's (actually the hobbit server manages creating the RRD's .. You just have to parse the information and send it in the proper format to hobbit).
That way you can have your multi-file RRD's (one per balancer or some) and you don't have to update to snapshot.
I wrote my script in perl (my very first perl script .. How I hate the language ;) and it works very well.
You will not find it in the Shire, mainly because I haven't posted it there yet.
If you need info on how the information is, and how to parse it, just drop me a line.
With regards,

Sigur ur Gu brandsson
Raf rninn ehf.
Su urhl   35
105 Reykjavik | Iceland
sigurdur at raforninn.is | www.raforninn.is
Office: +(354) 552 2070
Mobile: +(354) 867 3573

-----Original Message-----
From: Charles Goyard [mailto:charles.goyard at orange-ftgroup.com]
Sent: 10. okt ber 2007 06:50
To: hobbit at hswn.dk
Subject: Re: [hobbit] Custom graphs with multiple data sources
Hi,
Alan Sparks wrote :
...
However, with this, I am only able to manage one graph, and one data
source, per device.  I would like to graph all the session values.
I would also like to use multiple RRD files.  My experience with the
single-file, multiple-DS approach is that, if I add another virtual,
graphing will break (updates error out, probably since the RRD was
not created with this new data source defined).
[...]
Check the september 2007 mail archives, there's a discussion about it.
There's three solutions :

apply a patch I posted on the list (does split-ncv on a testname
basis). My patch has a memory leak, so beware (I kill my hobbitd_rrd
every 24h).
send "data" messages with a body like "foo.rrd:mydsname:GAUGE:500
(Sorry, no pointers for that one, and it needs a recent snapshot).
take a snapshot or a least the hobbitd/rrd directory from the source,
it includes the split-ncv feature without the memory leak (thank
Henrik).

--
Charles Goyard - charles.goyard at orange-ftgroup.com - (+33) 1 45 38 01
31 Orange Business Services - online multimedia  // ing nierie
To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
--
Alan Sparks, UNIX/Linux Systems Integration and Administration <asparks at doublesparks.net>

Sigurður Guðbrandsson
Raförninn ehf.
Suðurhlíð 35
105 Reykjavik | Iceland
sigurdur at raforninn.is | www.raforninn.is <http://www.raforninn.is/>
Office: +(354) 552 2070
Mobile: +(354) 867 3573

Custom graphs in hobbit, resend, ATTN Alan Sparks

sigurdur＠raforninn.is

This script reads digitemp values and writes them to a # temporary file which the hobbit client sends out to # the Hobbit Monitor.

This script really should be rewritten so it will contain subs for operations.

That would be much more tidier.

Written by Sigurdur Gudbrandsson

sigurdur at raforninn.is

Lets declare our variables

We need to get some variables from the environment, don't change this my $BB = $ENV{BB}; my $BBDISP = $ENV{BBDISP}; my $MACHINE = $ENV{MACHINE}; my $BBTMP = $ENV{BBTMP};

Change this to whatever you want your test to be named my $testname = "hitastig";

Don't change this or your test will never be green my $color = "green";

Set this to other than 0 if you want to debug my $debug = 0;

Lets connect to the database

Lets get the sensor ID's, names and min/max values

Now we have all the sensors in a two dimensional array, @sensors # Also, if there is a NULL value in min or max, it has been substituted with a value

Now we will get the temperature of each sensor from the database # and check if the temp is too high or too low.

We can close the database now

Next is to get our results into an array # For reference, 0=SerialNumber 1=name 2=description 3=min 4=max 5=value 6=color in our array

Now we decode from UTF-8 to latin1 (iso-8859-1) if there is anything in utf8 open(TEMP, ">$BBTMP/$testname.tmp.txt"); print TEMP $tmp; close(TEMP);

Now we get the converted data

Now finally we output the information to our beloved hobbit my $date = localtime; my $cmd = "$BB $BBDISP \"status $MACHINE.$testname $color $date \n $tmp \n \""; system($cmd);

Input parameters: Hostname, testname (column), and messagefile my $hostname=$ARGV[0]; my $testname=$ARGV[1]; my $fname=$ARGV[2];

print $buff[$i]."\n"; # Printing a test line to view the results

The next loop will print out the information for making the RRD file.

print "DS:".$key[$i].":GAUGE:600:0:100\n"; # Prints out each sensor name

This sub produces safe output so hobbit will be able to make the files # (It seems that hobbit doesn't like special letters in file names) sub makesafe {

$word =~ s/_//g;

$word = lc($word);