On Mon, Jan 24, 2005 at 03:14:54PM -0700, Charles Jones wrote:
How difficult is it to add custom graphs?
Well, it does require some programming - it's an advantage if you are familiar with C, since that is the language Hobbit is written in. I don't think it's terribly hard, but I am biased.
For example we have several Oracle standby databases that "resync" (import binary database changelogs) for several hours every night. I have a perl script which parses one of the logfiles created by this process, and gets values like the total time it took to import each binary log. Having a graph of the average time it takes to process would be VERY handy for metric and scaling evaluations.
The hard part usually is getting the data, and you have that already with your perl script.
Next step is getting the data to the Hobbit server. This is easy; decide on a unique name for the type of data you're handling - e.g. "orasync" - and use the "bb" utility to send it off as a "data" message to Hobbit. E.g. the following script runs your perl script, stores the output in a temporary file, and uses the "bb" utility from a Big Brother client installation to send this datafile to Hobbit in a "data" message:
#!/bin/sh
/foo/perlscript >/tmp/datafile
BBHOME=/usr/local/bbc export BBHOME . $BBHOME/etc/bbdef.sh
$BB $BBDISP "data $MACHINE.orasync
cat /tmp/datafile
"
Now the fun bit starts. Hobbit will automatically pass data-messages to all tasks monitoring the "data" channel. hobbitd_larrd is one of them, so you can either add some code to this "worker module", or you can create your own module from scratch using your favorite programming language. An example of a hobbitd worker module is in the hobbitd_sample.c application included with Hobbit.
Assuming you just add stuff to the existing hobbitd_larrd module, you must do two things:
Write a routine do_orasync_larrd() similar to the other ones in the hobbitd/larrd/*.c files, that receives the message, picks out the numbers that you want to store, and saves it in an RRD file;
Add a line to do_larrd.c at the end of the file, so when it sees the "orasync" message, it calls your do_orasync_larrd() routine.
You need to learn about RRDtool to really do this; the "rrdcreate" and "rrdgraph" manpages include some tips on how to define RRD's and how you can setup graphs.
Take a look at one of the simple ones, e.g. hobbitd/larrd/do_bbgen.c which picks out a single value from the status message that bbgen sends when updating the web-pages - the do_bbgen_larrd routine simply finds the string "TIME TOTAL <some number>", picks out the number (which is the time bbgen takes to generate the webpages and stores it in an RRD file.
The data stored in the RRD file is described in the bbgen_params variable (or "orasync_params" in your case):
static char *bbgen_params[] = { "rrdcreate", rrdfn, "DS:runtime:GAUGE:600:0:U", rra1, rra2, rra3, rra4, NULL };
The first and last line of this is static; you only change the "DS:..." line to define the data you store in the rrd.
When you have the data you want, put all of it in the "rrdvalues" string, with the timestamp in front, and call the create_and_update_rrd routine to do the work of saving the data.
So now you have an RRD file. Time to put it into hobbitgraph.cfg. This really depends on the kind of data you are handling.
The last step is to add "orasync" to the GRAPHS definition in hobbitserver.cfg. This causes the bb-larrdcolumn tool to include the orasync graph on the "trends" column page.
The first time you do it, it seems complex, and I admit: it isn't exactly trivial because there are many pieces that need to fit together. But once you get the first simple graph working you'll see that it isn't all that hard. And I'll be happy to help you if you run into problems along the way.
Henrik