Hi,
I wrote a script to get alert about CPU loads on a windows server using SNMP (no I don't want to / can't install the hobbit client on the server ;). The status is returned OK (green before 50%, orange from 50 to 75 and red over 75).
The thing is, there is no graph done (yet). I would like a graph that would show the load average variation.
(how) can the script be used to fill a RRD database ? Should the load average value be returned in a special variable ?
Thanks for help, Jo
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
joel at carnat.net wrote:
Hi,
I wrote a script to get alert about CPU loads on a windows server using SNMP (no I don't want to / can't install the hobbit client on the server ;). The status is returned OK (green before 50%, orange from 50 to 75 and red over 75).
The thing is, there is no graph done (yet).
Now that i'm approaching "working" with my ipmitool script, i'd like to remember how to do this too :D
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNlZxBvBcJFK6xYURArSbAJwMk6ZteKUTGA/CTXTLP4Cx8O+UQQCeMdsU nW5dJlJX51FUAFXdrJukqbs= =i9+u -----END PGP SIGNATURE-----
On 10/18/06, joel at carnat.net <joel at carnat.net> wrote:
Hi,
I wrote a script to get alert about CPU loads on a windows server using SNMP (no I don't want to / can't install the hobbit client on the server ;). The status is returned OK (green before 50%, orange from 50 to 75 and red over 75).
The thing is, there is no graph done (yet). I would like a graph that would show the load average variation.
(how) can the script be used to fill a RRD database ? Should the load average value be returned in a special variable ?
I'm doing something similar - SNMP queries against a Compaq system to produce a report like this:
CPU 1 min 5 min 30 min 60min &green 0 14% 9% 9% 9% &green 1 11% 7% 9% 9%
I changed the [rrdstatus] entry in server/etc/hobbitlaunch.cfg to include the --extra-script & --extra-tests entries as described in the hobbit_rrd man page.
If it gets through the mail system, my collection script is attached. It's not rocket science, but it seems to get the job done.
That pokes the data into an RRD, then it's just a question of setting up the graph config to show what you want to see.
Ralph Mitchell
On 10/18/06, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
On 10/18/06, joel at carnat.net <joel at carnat.net> wrote:
Hi,
I wrote a script to get alert about CPU loads on a windows server using SNMP (no I don't want to / can't install the hobbit client on the server ;). The status is returned OK (green before 50%, orange from 50 to 75 and red over 75).
The thing is, there is no graph done (yet). I would like a graph that would show the load average variation.
(how) can the script be used to fill a RRD database ? Should the load average value be returned in a special variable ?
I'm doing something similar - SNMP queries against a Compaq system to produce a report like this:
CPU 1 min 5 min 30 min 60min &green 0 14% 9% 9% 9% &green 1 11% 7% 9% 9%
I changed the [rrdstatus] entry in server/etc/hobbitlaunch.cfg to include the --extra-script & --extra-tests entries as described in the hobbit_rrd man page.
If it gets through the mail system, my collection script is attached. It's not rocket science, but it seems to get the job done.
That pokes the data into an RRD, then it's just a question of setting up the graph config to show what you want to see.
Ralph Mitchell
The script was blocked, so here it is:
Ralph Mitchell
#!/bin/sh
Input parameters: Hostname, testname (column), and messagefile
Messagefile contains this:
Thu Jan 12 06:33:28 CST 2006 compaq_cpu
This checkout shows CPU utilization as a percentage of the
theoretical maximum over 1min, 5min, 30min & 60min periods.
CPU 1 min 5 min 30 min 60min
&green 0 7% 7% 6% 7%
&green 1 7% 9% 8% 10%
TMPLOG="/tmp/$1.$2.out" echo "$1, $2, $3" > $TMPLOG cat $3 >> $TMPLOG
HOSTNAME="$1" TESTNAME="$2" FNAME="$3"
Check the test name so that this script can service multiple
data collection needs
if [ "$TESTNAME" = "cpqcpu" ] then
The RRD dataset definitions
echo "DS:1min:GAUGE:600:0:100" echo "DS:5min:GAUGE:600:0:100" echo "DS:30min:GAUGE:600:0:100" echo "DS:60min:GAUGE:600:0:100"
Analyze the message we got
cat $FNAME | while read line
do
if [ "$line" ]; then
line=echo $line | sed -e 's/&//g' -e 's/%//g'
set $line
case $1 in
green)
echo "cpqcpu.$2.rrd"
echo "$3:$4:$5:$6"
echo "cpqcpu.$2.rrd" >> $TMPLOG
echo "$3:$4:$5:$6" >> $TMPLOG
;;
yellow)
echo "cpqcpu.$2.rrd"
echo "$3:$4:$5:$6"
echo "cpqcpu.$2.rrd" >> $TMPLOG
echo "$3:$4:$5:$6" >> $TMPLOG
;;
red)
echo "cpqcpu.$2.rrd"
echo "$3:$4:$5:$6"
echo "cpqcpu.$2.rrd" >> $TMPLOG
echo "$3:$4:$5:$6" >> $TMPLOG
;;
*)
;;
esac
fi
done
fi
exit 0
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
joel at carnat.net wrote
(how) can the script be used to fill a RRD database?
Okay, i'm *way* behind you guys here. Where's the best starting point for whipping up info from scratch?
Of note, this is what i wound up with. Yes, wince away at all those pipes. It isn't zippy, but this Rube Goldbergesque script does work.
Caveats: add hobbit to your ipmi users (ipmitool makes this easy), and make sure it can read the /dev/ipmi0, or you're going to feel silly (i made an ipmi group and a udev permission rule to keep the device in it).
Any advice on the shortening the awksed acrobatics below would be appreciated. The problem is that my raw output does NOT include "CPU" or similar.. just "temp." Looking it up, line 1 is the external CPU temp and line 2 is internal... but it comes out like this:
hobbit at currant:~$ ipmitool sensor Temp | 31.000 | degrees C | ok | na | 5.000 | 10.000 | 85.000 | 90.000 | na Temp | 38.000 | degrees C | ok | na | 5.000 | 10.000 | 85.000 | 90.000 | na Planar Temp | 28.000 | degrees C | ok | na | 5.000 | 10.000 | 70.000 | 75.000 | na VRD 0 Temp | 22.000 | degrees C | ok | na | 5.000 | 10.000 | 65.000 | 70.000 | na VRD 1 Temp | 24.000 | degrees C | ok | na | 5.000 | 10.000 | 65.000 | 70.000 | na
"Temp" indeed! You have to "just know" what the first two lines are. This is the start of my using this expensive BMC :), i plan on adding everything Systems wants to see such as fan speed, general case temp, RAID assertions and other joys. For now, getting and graphing CPU core temp is my practice target. here's the script:
#!/bin/sh
umask 133
COLUMN=temp
COLOR=green
MSG="Current Temp"
host=hostname
ipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'
Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Very Bad.
if temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot!
"else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain.
"fi
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR date
${MSG} "
exit 0
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNmG9BvBcJFK6xYURAoSoAJ9wQS6sFNM5/Q+DFZi27AvzSYTA5gCdGntj gk6ZqNKnmGE2tR8bmS+MpeE= =jO7X -----END PGP SIGNATURE-----
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
ipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=
tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'
Is there a compelling reason not to fold the above into one line:
temp=`ipmitool sensor | tail -n 4 | head -n 1 | awk '{ print $3 }'`Ok, we have the temp in degrees as an integer.
Now to see if it's Very Bad.
if temp > 130 then
Does this bit work?? I get:
./ipmi.sh: line 22: temp: command not foundand an empty file called '130'. This works for me:
if [ $temp -gt 130 ]
thenThe simplest way to graph it would be to have the message read something like:
cpu temp : $tempthen follow the instructions in the "Custom Graphs" entry from the Help dropdown. You already have the first part - collecting the data. If you use NCV, the above line would go into the RRD as cputemp, because it takes everything before the colon and strips out spaces to make the name, then takes the rest of the line as the number. That probably a gross over-simplification... :)
Ralph
On 10/18/06, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
ipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=
tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'Is there a compelling reason not to fold the above into one line:
temp=`ipmitool sensor | tail -n 4 | head -n 1 | awk '{ print $3 }'`
Wups, my bad... Make that:
cpu_temp=`ipmitool sensor | tail -n 4 | head -n 1 | awk '{ print$3 }' | sed -e 's/\..*//'`
Ralph
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Okay,
I have made the changes needed to *show* the graph, and i have my script. What i can't figure out is how to get the data from this client script onto the server, in data/rrd/testname, so it can be read.
I've gone over the manpages and i'm still confused, sorry. Do i need to call another channel with the --extra-tests option? Is it in a config file somewhere? The sample script of "weather in Copenhagen" is a little opaque, it just shows a bunch of echoes but not their context. Is THAT script called by another within hobbit? Lastly is there a howto somewhere for exporting one's custom script data to RRD?
TIA,
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNm6gBvBcJFK6xYURAhIUAKCDQrXdO2M3AeJ49Kyc3yXe0+6HfACfT1tx uSo+W+hmwSM+AapWUMsJzCg= =8Pr5 -----END PGP SIGNATURE-----
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
I've gone over the manpages and i'm still confused, sorry. Do i need to call another channel with the --extra-tests option? Is it in a config file somewhere? The sample script of "weather in Copenhagen" is a little opaque, it just shows a bunch of echoes but not their context. Is THAT script called by another within hobbit? Lastly is there a howto somewhere for exporting one's custom script data to RRD?
When you add the --extra-script option in the [rrdstatus] block, what you're doing is handing the hobbit_rrd program another way to process information. Your script is fired up with the hostname, column name and the name of a file containing the log message, all on the command line. The hobbit_rrd then watches the script's standard output for dataset definitions (in case the RRD doesn't exist yet), then a number of colon-separated values to stuff into the RRD. hobbit_rrd takes care of the RRD creation and stuffing, you just need to echo out the correct format.
It took me several iterations to get this straight... :)
Ralph
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
I've gone over the manpages and i'm still confused, sorry. Do i need to call another channel with the --extra-tests option? Is it in a config file somewhere? The sample script of "weather in Copenhagen" is a little opaque, it just shows a bunch of echoes but not their context. Is THAT script called by another within hobbit? Lastly is there a howto somewhere for exporting one's custom script data to RRD?
When you add the --extra-script option in the [rrdstatus] block,
Well i'm stuck already. What file is that block in? I can't find it in 'server.cfg nor 'graph.cfg...
hobbit_rrd takes care of the RRD creation and stuffing, you just need to echo out the correct format.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all?
It took me several iterations to get this straight... :)
I bet i beat you! :D
Ralph
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNnaKBvBcJFK6xYURApsoAJwIKXUJGbo5anfA1yGRedsQQ2h36ACeLL0K 1NBZFAPuTCzlw0y51dblKlY= =wl4p -----END PGP SIGNATURE-----
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block,
Well i'm stuck already. What file is that block in? I can't find it in 'server.cfg nor 'graph.cfg...
It's in server/etc/hobbitlaunch.cfg
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all?
That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
It took me several iterations to get this straight... :)
I bet i beat you! :D
Ahh, if this wasn't a family channel... :)
Note that if you want to push several different columns through this script, you'll need something that distinguishes between them, then echo out appropriate RRD names/data. The hobbitd_rrd program only runs *one* extra-script to deal with any results you send it...
So, if you're planning on several columns - cputemp, planartemp, vrd0temp, etc, you'd need to allow for that in the script:
if [ "$2" = "cputemp"}; then ... elif [ "$2" = "planartemp" ]; then ... elif [ "$2" = "vrd0temp" ]; then ... fi
On the other hand, if you're planning on pushing out all that stuff in the same status message, you probably should go ahead and add the extra RRD creation rows right away, and simply echo out zeroes for the values until you figure out how to extract them:
echo "DS:cputemp:GAUGE:600:0:U"
echo "DS:planartemp:GAUGE:600:0:U"
echo "DS:vrd0temp:GAUGE:600:0:U"
echo "cputemp.rrd"
echo "$temp:0:0"I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
Ralph
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block,
Well i'm stuck already. What file is that block in? I can't find it in 'server.cfg nor 'graph.cfg...
It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all?
That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
"Sir, I have no data."
Ok, so i did this
[cputemp] TITLE CPU Temp YAXIS Degrees Fahrenheit DEF:avg=cputemp.rrd:cputemp:AVERAGE AREA:cputemp#00CC00:CPU Temp Average -u 1.0 GPRINT:cputemp:LAST: \: %5.11f (cur) GPRINT:cputemp:MAX: \: %5.1lf (max) GPRINT:cputemp:MIN: \: %5.1lf (min) GPRINT:cputemp:AVERAGE: \: %5.1lf (avg)\n
and this
GRAPHS="la,disk,inode,qtree,memory,users,cputemp,.....8<...
as well, on the server. but what i get is a text link called
Status unchanged in 1 hours, 41 minutes Status message received from 10.10.10.12 hobbit graph cputemp
that leads to nothing. Not even a "nan" graph :P
I think maybe by echo is in the wrong place...? Should it be before or after the $BB msg command? Does it matter?
hobbit at currant:~/client/ext$ cat cputemp.sh #!/bin/sh
umask 133
COLUMN=cputemp
COLOR=green
MSG="Current CPU Temperature: "
host=hostname
ipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'
Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Bad, or even Very Bad.
if $temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot!
"else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain.
"fi
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd" echo "$temp"
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR date
${MSG} "
exit 0
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNnyrBvBcJFK6xYURAt3KAJ9TmnWy1GLkx/yfDFB4G4QBEtPVCQCfYp1l Rt0EeHF8jWGfJnjxzUtX/ss= =xFd4 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Rob Munsch wrote:
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block, Well i'm stuck already. What file is that block in? I can't find it in 'server.cfg nor 'graph.cfg... It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all? That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
yeah, i'm kind of an idiot:
/local/hobbit/client/ext/cputemp.sh: line 25: 100: command not found DS:cputemp:GAUGE:600:0:U cputemp.rrd 100 /local/hobbit/client/ext/cputemp.sh: line 25: 100: command not found DS:cputemp:GAUGE:600:0:U cputemp.rrd 100
On the client my script is happily shouting this out to stdout. Where do i include these "echo" lines to get them properly noted by the server's rrdtool? I can see that no cputemp.rrd exists on the server. It's not getting there.
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNn3FBvBcJFK6xYURAitiAJ9r6yMc6JTVvCHBvgVlq4/bWeQseACfdCiD m3BEriuMmaDMZMGqxxhweeA= =MT36 -----END PGP SIGNATURE-----
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block,
Well i'm stuck already. What file is that block in? I can't find it in 'server.cfg nor 'graph.cfg...
It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all?
That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
"Sir, I have no data."
Ok, so i did this
[cputemp] TITLE CPU Temp YAXIS Degrees Fahrenheit DEF:avg=cputemp.rrd:cputemp:AVERAGE AREA:cputemp#00CC00:CPU Temp Average -u 1.0 GPRINT:cputemp:LAST: \: %5.11f (cur) GPRINT:cputemp:MAX: \: %5.1lf (max) GPRINT:cputemp:MIN: \: %5.1lf (min) GPRINT:cputemp:AVERAGE: \: %5.1lf (avg)\n
and this
GRAPHS="la,disk,inode,qtree,memory,users,cputemp,.....8<...
as well, on the server. but what i get is a text link called
Status unchanged in 1 hours, 41 minutes Status message received from 10.10.10.12 hobbit graph cputemp
that leads to nothing. Not even a "nan" graph :P
I think maybe by echo is in the wrong place...? Should it be before or after the $BB msg command? Does it matter?
hobbit at currant:~/client/ext$ cat cputemp.sh #!/bin/sh
umask 133
COLUMN=cputemp COLOR=green MSG="Current CPU Temperature: " host=
hostnameipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=
tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Bad, or even Very Bad.
if $temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot! "else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain. "fi
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd" echo "$temp"
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR
date${MSG} "
exit 0
There should be *two* scripts... The data collecting script that produces the report and uses "$BB BBDISP....." to send the data to Hobbit, and a separate script that echoes out the RRD creation bits and pieces. The first one would be launched by hobbit or cron or some other means, and the second is executed by hobbit_rrd due to the --extra-script flag in '[rrdstatus]'. *That's* the one where the "echo $temp" goes...
Ralph
Rob,
You are trying to combine two scripts into one. Read the FAQ.
GLH
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 2:13 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block,
Well i'm stuck already. What file is that block in? I can't find it
in 'server.cfg nor 'graph.cfg...
It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all?
That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
"Sir, I have no data."
Ok, so i did this
[cputemp] TITLE CPU Temp YAXIS Degrees Fahrenheit DEF:avg=cputemp.rrd:cputemp:AVERAGE AREA:cputemp#00CC00:CPU Temp Average -u 1.0 GPRINT:cputemp:LAST: \: %5.11f (cur) GPRINT:cputemp:MAX: \: %5.1lf (max) GPRINT:cputemp:MIN: \: %5.1lf (min) GPRINT:cputemp:AVERAGE: \: %5.1lf (avg)\n
and this
GRAPHS="la,disk,inode,qtree,memory,users,cputemp,.....8<...
as well, on the server. but what i get is a text link called
Status unchanged in 1 hours, 41 minutes Status message received from 10.10.10.12 hobbit graph cputemp
that leads to nothing. Not even a "nan" graph :P
I think maybe by echo is in the wrong place...? Should it be before or after the $BB msg command? Does it matter?
hobbit at currant:~/client/ext$ cat cputemp.sh #!/bin/sh
umask 133
COLUMN=cputemp
COLOR=green
MSG="Current CPU Temperature: "
host=hostname
ipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'
Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Bad, or even Very Bad.
if $temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot!
"else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain.
"fi
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd" echo "$temp"
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR date
${MSG} "
exit 0
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNnyrBvBcJFK6xYURAt3KAJ9TmnWy1GLkx/yfDFB4G4QBEtPVCQCfYp1l Rt0EeHF8jWGfJnjxzUtX/ss= =xFd4 -----END PGP SIGNATURE-----
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
You are trying to combine two scripts into one. Read the FAQ.
I don't faintly understand what you mean. I have one script.
And as stated, i have both read the FAQ and the manpages and still not gotten it to click in my head; hence, i am here.
GLH
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 2:13 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block, Well i'm stuck already. What file is that block in? I can't find it
in 'server.cfg nor 'graph.cfg... It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all? That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
"Sir, I have no data."
Ok, so i did this
[cputemp] TITLE CPU Temp YAXIS Degrees Fahrenheit DEF:avg=cputemp.rrd:cputemp:AVERAGE AREA:cputemp#00CC00:CPU Temp Average -u 1.0 GPRINT:cputemp:LAST: \: %5.11f (cur) GPRINT:cputemp:MAX: \: %5.1lf (max) GPRINT:cputemp:MIN: \: %5.1lf (min) GPRINT:cputemp:AVERAGE: \: %5.1lf (avg)\n
and this
GRAPHS="la,disk,inode,qtree,memory,users,cputemp,.....8<...
as well, on the server. but what i get is a text link called
Status unchanged in 1 hours, 41 minutes Status message received from 10.10.10.12 hobbit graph cputemp
that leads to nothing. Not even a "nan" graph :P
I think maybe by echo is in the wrong place...? Should it be before or after the $BB msg command? Does it matter?
hobbit at currant:~/client/ext$ cat cputemp.sh #!/bin/sh
umask 133
COLUMN=cputemp COLOR=green MSG="Current CPU Temperature: " host=
hostnameipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=
tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Bad, or even Very Bad.
if $temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot! "else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain. "fi
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd" echo "$temp"
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR
date${MSG} "
exit 0
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNoHPBvBcJFK6xYURAnmeAJ99en7G3Pb0BqRK1WJMtJYmK6fpVACfbajb XocUwudUBdzgQLeAeQITUL0= =4a/u -----END PGP SIGNATURE-----
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
You are trying to combine two scripts into one. Read the FAQ.
I don't faintly understand what you mean. I have one script.
Yes, but there should be two scripts.
Script #1: runs anywhere - could be on your Hobbit server, could be on one or more remote server(s) at any random location on the planet. It reads the sensors, extracts the relevant data, formats the message and sends it to the Hobbit server
Script #2: is executed by hobbitd_rdd via the --extra-script option. This one extracts relevant information from the incoming status report and echoes out the RRD creation parameters, the RRD file name, and the data to stuff into the RRD. Hobbit_rrd reads that and does what's necessary to save the data in an RRD.
Ralph Mitchell
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
You are trying to combine two scripts into one. Read the FAQ.
I don't faintly understand what you mean. I have one script.
Yes, but there should be two scripts.
Script #1: runs anywhere - could be on your Hobbit server, could be on one or more remote server(s) at any random location on the planet. It reads the sensors, extracts the relevant data, formats the message and sends it to the Hobbit server
Script #2: is executed by hobbitd_rdd via the --extra-script option. This one extracts relevant information from the incoming status report and echoes out the RRD creation parameters, the RRD file name, and the data to stuff into the RRD. Hobbit_rrd reads that and does what's necessary to save the data in an RRD.
This is no way clearly explained by the docs if you didn't already know it. it indicates that your One True Custom Script lives on your client in the case of a client-side script, and sends info to the server as it's called.
It goes on to describe changes to be made to various .cfg files, which i *have made*, so this data gets processed.
The docs indicate that --extra-script should have "=<myclientsidescriptname>" tacked on to it, which i have also done.
if this is not the case, please enlighten, and when i am done with this i would happy to provide clarification to the documentation.
Ralph Mitchell
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNofpBvBcJFK6xYURAr+HAJsGoHHjZHEmJVUsg2HdoCZoTixl8gCghacd mxRTo7hil30CyNhMFTOdi6Q= =WGAS -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Script #2: is executed by hobbitd_rdd via the --extra-script option. This one extracts relevant information from the incoming status report and echoes out the RRD creation parameters, the RRD file name, and the data to stuff into the RRD. Hobbit_rrd reads that and does what's necessary to save the data in an RRD.
maybe i'm not being clear. It's been a long day...
I understand that the goal is to get data to RRD in a format it likes with data and filenames it understands. That part is obvious. The concept is not the issue; i know i want to graph things!
What is not obvious is where to warn hobbit that this info is coming and precisely how to send that info; "echo" clientside was, i thought, read by the client's hobbitd as in a wrapper, but i guess not..?
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNoh/BvBcJFK6xYURAsnbAJ4thkw0rNR4Jvn7WoeX3s5ujdenygCfcz0J HwEES1jq8lQLA8s9YFD/04w= =kA5p -----END PGP SIGNATURE-----
Rob,
The problem is that this is scattered across several different places in the Hobbit architecture.
- Detecting the numbers. In a client-side (or server-side, though more rare) script, you do whatever you need to do to get some numbers. You format these numbers and send them to Hobbit. All Hobbit is looking for is the header line where you give it a color. The rest is just text that Hobbit will faithfully store and repeat back to you as date under the column you have selected. Knowing that only the first line is important, then you can do WHATEVER you want to show your numbers. Knowing that you will be parsing this text when you receive it, it is very helpful to put the data in a format that is easy to recognize and parse later, but is still readable to the human eye.
Another note -- you need to do all of this on new columns -- don't try to augment the existing Hobbit tests.
To debug: simply eyeball the text that is listed as column detail for your test. If Hobbit is getting it, and assigning a color, then you are out of the gate.
- On the Hobbit server, you write a custom parser script. This script is called with three parameters -- the column name (aka test name), the node name associated with the test, and a file name. What your script should do is branch appropriately according to the test name, and then read the file, parsing it as needed. The script can be written in any language (Perl comes to mind if you don't like shell). But keep in mind that there is ONLY ONE parsing script allowed, and it needs to be able to service all of your custom tests.
When you find the numbers you want for your test, you simply write to STDOUT the sequence that Hobbit is looking for to put your data in RRDS. ANYTHING else that you want to say in your script needs to be directed to a log file, or some other place.
You have to tell the Hobbit server to use your script -- this requires modification of the hobbitlaunch file to change the parameter list for the hobbit_rrd modules. Notice that there are two of them -- one for data, and one for status.
To debug: run your new test script by hand, supplying a bogus node name, the name of the real test, and the name of a file. I found it helpful to snag all of the test from the column detail and save it in a file, and then use that file as the file parameter. All you should see come out of your script is the RRD language -- first the DS statements, then the file name, and finally the data record. You can write it all to one RRD, or to several, depending on what you write out.
- Now that RRD's are being created, you have to go and create graph definitions in hobbit-graph.cfg and you have to tell Hobbit about them so you can see them. You can choose from the wealth that is already there to get examples or "borrow" code. You will need to consult the RRDTool documentation to understand these graph definitions, because Hobbit graph is simply passing these definitions through to RRDTool for graphing.
Now you need to update the Hobbit server configuration to include your graphs in both lists -- TRENDS and TEST2RRD. I could not keep things straight, so I ended up calling my graph definitions by the same name as my tests. So for test "foo" I create a graph definition "foo" instead of "bar".
To debug: First, are the RRD entries being created? You should see them in the data directory underneath the name of the host where the client-side test is being run.
Next, is there data being written to the RRD? You can dump the contents with RRDTool, but I forget the syntax. All you are looking for is that your data is ending up in the right place.
Next, can you see your graphs? If you don't see a graph at all on your test page, then you probably have not identified it correctly to the Hobbit server in the hobbitconfig. If you see a broken link where you expect a graph, the problem is probably with your hobbitgraph.cfg file -- you made an error specifying the graph.
I have created 7 or 8 custom tests with graphs and everything. The first one took several days, then I got a lot faster.
I wish I had time to be more exhaustive in the instructions...
Good luck!
GLH
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 3:03 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Script #2: is executed by hobbitd_rdd via the --extra-script option. This one extracts relevant information from the incoming status report
and echoes out the RRD creation parameters, the RRD file name, and the
data to stuff into the RRD. Hobbit_rrd reads that and does what's necessary to save the data in an RRD.
maybe i'm not being clear. It's been a long day...
I understand that the goal is to get data to RRD in a format it likes with data and filenames it understands. That part is obvious. The concept is not the issue; i know i want to graph things!
What is not obvious is where to warn hobbit that this info is coming and precisely how to send that info; "echo" clientside was, i thought, read by the client's hobbitd as in a wrapper, but i guess not..?
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNoh/BvBcJFK6xYURAsnbAJ4thkw0rNR4Jvn7WoeX3s5ujdenygCfcz0J HwEES1jq8lQLA8s9YFD/04w= =kA5p -----END PGP SIGNATURE-----
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
Hi greg. thanks for your patience.
The problem is that this is scattered across several different places in the Hobbit architecture.
and how XD
- Detecting the numbers. In a client-side (or server-side, though more rare) script, you do whatever you need to do to get some numbers. You format these numbers and send them to Hobbit. All Hobbit is looking for is the header line where you give it a color. The rest is just text that Hobbit will faithfully store and repeat back to you as date under the column you have selected. Knowing that only the first line is important, then you can do WHATEVER you want to show your numbers. Knowing that you will be parsing this text when you receive it, it is very helpful to put the data in a format that is easy to recognize and parse later, but is still readable to the human eye.
This i got...
Another note -- you need to do all of this on new columns -- don't try to augment the existing Hobbit tests.
...and did. And works well. I have a new CPUTemp column which shows the "CPU Temp: X degrees Fahrenheit." That was the easy part.
- On the Hobbit server, you write a custom parser script. This script is called with three parameters -- the column name (aka test name), the node name associated with the test, and a file name. What your script should do is branch appropriately according to the test name, and then read the file, parsing it as needed. The script can be written in any language (Perl comes to mind if you don't like shell). But keep in mind that there is ONLY ONE parsing script allowed, and it needs to be able to service all of your custom tests.
Oho. I have NO new script on the server. This part is what isn't documented well i think... is it a copy of my client script? Does it pull from the data that the client script is sending to the server? This what seems odd to me, the client is already sending the numbers i need to generate the color status. Why can't i use that??
When you find the numbers you want for your test, you simply write to STDOUT the sequence that Hobbit is looking for to put your data in RRDS.
which is where those Echo commands came from, i see now.
ANYTHING else that you want to say in your script needs to be directed to a log file, or some other place.
You have to tell the Hobbit server to use your script -- this requires modification of the hobbitlaunch file to change the parameter list for the hobbit_rrd modules. Notice that there are two of them -- one for data, and one for status.
Did both of those. They just aren't uh running anything atm XD
To debug: run your new test script by hand, supplying a bogus node name,
node name? :-/
the name of the real test, and the name of a file. I found it helpful to snag all of the test from the column detail and save it in a file, and then use that file as the file parameter. All you should see come out of your script is the RRD language -- first the DS statements, then the file name, and finally the data record. You can write it all to one RRD, or to several, depending on what you write out.
- Now that RRD's are being created, you have to go and create graph definitions in hobbit-graph.cfg
This is also already done, since i thought i'd taken care of the rest.
Now you need to update the Hobbit server configuration to include your graphs in both lists -- TRENDS and TEST2RRD. I could not keep things straight, so I ended up calling my graph definitions by the same name as my tests. So for test "foo" I create a graph definition "foo" instead of "bar".
Done also.
To debug: First, are the RRD entries being created? You should see them in the data directory underneath the name of the host where the client-side test is being run.
Next, is there data being written to the RRD? You can dump the contents with RRDTool, but I forget the syntax. All you are looking for is that your data is ending up in the right place.
Next, can you see your graphs? If you don't see a graph at all on your test page, then you probably have not identified it correctly to the Hobbit server in the hobbitconfig. If you see a broken link where you expect a graph, the problem is probably with your hobbitgraph.cfg file -- you made an error specifying the graph.
argh. not done well, apparently... instead of what one would expect with correct definitions - a nan or blank graph - i have simply
hobbit graph ncv:cputemp
where the graph should be. In the absence of data, i should still have a blank graph, if i defined it correctly, right? So i messed that up.
I have created 7 or 8 custom tests with graphs and everything. The first one took several days, then I got a lot faster.
I shall continue to slug away.
I wish I had time to be more exhaustive in the instructions...
Thank you for your time, this is starting to make sense to me now.
I still do not quite get how the actual data gets to rrd. It seems to me that there is an isolated client script that sends data which determines the status color.. then a DIFFERENT server-side script is used to send the SAME DATA to RRD for graphing. ?!?! yesno?
Why isn't there a way to use the existing data that the client's sending for graphing... for CPU, MEM, etc. isn't that already happening? or have i failed another concept here.
On the server, i have made all config changes (despite the badly-crafted graph definition) but there is NO rrd data collected. I still don't get the client-server communication, i guess.
Does anyone have a sample of a server-side RRD data collection script i could borrow, so i see how and where from it's grabbed? Thanks...
Good luck!
i don't need luck, i need to stop being so thick... :)
GLH
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 3:03 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Script #2: is executed by hobbitd_rdd via the --extra-script option. This one extracts relevant information from the incoming status report
and echoes out the RRD creation parameters, the RRD file name, and the
data to stuff into the RRD. Hobbit_rrd reads that and does what's necessary to save the data in an RRD.
maybe i'm not being clear. It's been a long day...
I understand that the goal is to get data to RRD in a format it likes with data and filenames it understands. That part is obvious. The concept is not the issue; i know i want to graph things!
What is not obvious is where to warn hobbit that this info is coming and precisely how to send that info; "echo" clientside was, i thought, read by the client's hobbitd as in a wrapper, but i guess not..?
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNpSUBvBcJFK6xYURArUzAJ9l4tNis+OwO0PnLrhADh1NUgaCIgCeMwC8 7yWkzgJoIXM+BvxSmoxDFQA= =/rmJ -----END PGP SIGNATURE-----
Contact me offline and I fix you up.
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 3:55 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
Hi greg. thanks for your patience.
The problem is that this is scattered across several different places in the Hobbit architecture.
and how XD
- Detecting the numbers. In a client-side (or server-side, though more rare) script, you do whatever you need to do to get some numbers. You
format these numbers and send them to Hobbit. All Hobbit is looking for is the header line where you give it a color. The rest is just text that Hobbit will faithfully store and repeat back to you as date under the column you have selected. Knowing that only the first line is important, then you can do WHATEVER you want to show your numbers. Knowing that you will be parsing this text when you receive it, it is very helpful to put the data in a format that is easy to recognize and
parse later, but is still readable to the human eye.
This i got...
Another note -- you need to do all of this on new columns -- don't try
to augment the existing Hobbit tests.
...and did. And works well. I have a new CPUTemp column which shows the "CPU Temp: X degrees Fahrenheit." That was the easy part.
- On the Hobbit server, you write a custom parser script. This script is called with three parameters -- the column name (aka test name), the node name associated with the test, and a file name. What your script should do is branch appropriately according to the test name, and then read the file, parsing it as needed. The script can be written in any language (Perl comes to mind if you don't like shell).
But keep in mind that there is ONLY ONE parsing script allowed, and it
needs to be able to service all of your custom tests.
Oho. I have NO new script on the server. This part is what isn't documented well i think... is it a copy of my client script? Does it pull from the data that the client script is sending to the server? This what seems odd to me, the client is already sending the numbers i need to generate the color status. Why can't i use that??
When you find the numbers you want for your test, you simply write to STDOUT the sequence that Hobbit is looking for to put your data in RRDS.
which is where those Echo commands came from, i see now.
ANYTHING else that you want to say in your script needs to be directed
to a log file, or some other place.
You have to tell the Hobbit server to use your script -- this requires
modification of the hobbitlaunch file to change the parameter list for
the hobbit_rrd modules. Notice that there are two of them -- one for data, and one for status.
Did both of those. They just aren't uh running anything atm XD
To debug: run your new test script by hand, supplying a bogus node name,
node name? :-/
the name of the real test, and the name of a file. I found it helpful
to snag all of the test from the column detail and save it in a file, and then use that file as the file parameter. All you should see come
out of your script is the RRD language -- first the DS statements, then the file name, and finally the data record. You can write it all
to one RRD, or to several, depending on what you write out.
- Now that RRD's are being created, you have to go and create graph definitions in hobbit-graph.cfg
This is also already done, since i thought i'd taken care of the rest.
Now you need to update the Hobbit server configuration to include your graphs in both lists -- TRENDS and TEST2RRD. I could not keep things straight, so I ended up calling my graph definitions by the same name as my tests. So for test "foo" I create a graph definition "foo" instead of "bar".
Done also.
To debug: First, are the RRD entries being created? You should see them in the data directory underneath the name of the host where the client-side test is being run.
Next, is there data being written to the RRD? You can dump the contents with RRDTool, but I forget the syntax. All you are looking for is that your data is ending up in the right place.
Next, can you see your graphs? If you don't see a graph at all on your test page, then you probably have not identified it correctly to the Hobbit server in the hobbitconfig. If you see a broken link where
you expect a graph, the problem is probably with your hobbitgraph.cfg file -- you made an error specifying the graph.
argh. not done well, apparently... instead of what one would expect with correct definitions - a nan or blank graph - i have simply
hobbit graph ncv:cputemp
where the graph should be. In the absence of data, i should still have a blank graph, if i defined it correctly, right? So i messed that up.
I have created 7 or 8 custom tests with graphs and everything. The first one took several days, then I got a lot faster.
I shall continue to slug away.
I wish I had time to be more exhaustive in the instructions...
Thank you for your time, this is starting to make sense to me now.
I still do not quite get how the actual data gets to rrd. It seems to me that there is an isolated client script that sends data which determines the status color.. then a DIFFERENT server-side script is used to send the SAME DATA to RRD for graphing. ?!?! yesno?
Why isn't there a way to use the existing data that the client's sending for graphing... for CPU, MEM, etc. isn't that already happening? or have i failed another concept here.
On the server, i have made all config changes (despite the badly-crafted graph definition) but there is NO rrd data collected. I still don't get the client-server communication, i guess.
Does anyone have a sample of a server-side RRD data collection script i could borrow, so i see how and where from it's grabbed? Thanks...
Good luck!
i don't need luck, i need to stop being so thick... :)
GLH
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 3:03 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Script #2: is executed by hobbitd_rdd via the --extra-script option. This one extracts relevant information from the incoming status report
and echoes out the RRD creation parameters, the RRD file name, and the
data to stuff into the RRD. Hobbit_rrd reads that and does what's necessary to save the data in an RRD.
maybe i'm not being clear. It's been a long day...
I understand that the goal is to get data to RRD in a format it likes with data and filenames it understands. That part is obvious. The concept is not the issue; i know i want to graph things!
What is not obvious is where to warn hobbit that this info is coming and precisely how to send that info; "echo" clientside was, i thought,
read by the client's hobbitd as in a wrapper, but i guess not..?
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNpSUBvBcJFK6xYURArUzAJ9l4tNis+OwO0PnLrhADh1NUgaCIgCeMwC8 7yWkzgJoIXM+BvxSmoxDFQA= =/rmJ -----END PGP SIGNATURE-----
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
I still do not quite get how the actual data gets to rrd. It seems to me that there is an isolated client script that sends data which determines the status color.. then a DIFFERENT server-side script is used to send the SAME DATA to RRD for graphing. ?!?! yesno?
Yep.
Why isn't there a way to use the existing data that the client's sending for graphing... for CPU, MEM, etc. isn't that already happening? or have i failed another concept here.
I think it is using the existing data... If the incoming report looks like it should be graphed (via TRENDS in bb-hosts, I think) it gets shunted to the hobbit_rrd program which does nothing if it finds none of the standard data (uptime, df, vmstat, etc). That's where the --extra-script script comes in. It breaks up the message into a form that hobbit_rrd can read. hobbit_rrd does something like fork-and-execute to run the extra script, and listens to the output channel from the script. It expects to see the RRD creation formula, the RRD name, then an RRD data string. hobbit_rrd then does whatever magic is required to stuff the data into the RRD file.
On the server, i have made all config changes (despite the badly-crafted graph definition) but there is NO rrd data collected. I still don't get the client-server communication, i guess.
Does anyone have a sample of a server-side RRD data collection script i could borrow, so i see how and where from it's grabbed? Thanks...
This is a hobbit_rrd script I use to pick up numbers from a cpu load report gathered from Compaq servers via an SNMP query. The line
if [ "$TESTNAME" = "cpqcpu" ]
allows the same script to process reports from multiple different column names. In this particular case, the column is 'cpqcpu'.
Rlaph Mitchell
#!/bin/sh
Input parameters: Hostname, testname (column), and messagefile
Messagefile contains this:
Thu Jan 12 06:33:28 CST 2006 compaq_cpu
This checkout shows CPU utilization as a percentage of the
theoretical maximum over 1min, 5min, 30min & 60min periods.
CPU 1 min 5 min 30 min 60min
&green 0 7% 7% 6% 7%
&green 1 7% 9% 8% 10%
TMPLOG="/tmp/$1.$2.out" echo "$1, $2, $3" > $TMPLOG cat $3 >> $TMPLOG
HOSTNAME="$1" TESTNAME="$2" FNAME="$3"
Check the test name so that this script can service multiple
data collection needs
if [ "$TESTNAME" = "cpqcpu" ] then
The RRD dataset definitions
echo "DS:1min:GAUGE:600:0:100" echo "DS:5min:GAUGE:600:0:100" echo "DS:30min:GAUGE:600:0:100" echo "DS:60min:GAUGE:600:0:100"
Analyze the message we got
cat $FNAME | while read line
do
if [ "$line" ]; then
line=echo $line | sed -e 's/&//g' -e 's/%//g'
set $line
case $1 in
green)
echo "cpqcpu.$2.rrd"
echo "$3:$4:$5:$6"
echo "cpqcpu.$2.rrd" >> $TMPLOG
echo "$3:$4:$5:$6" >> $TMPLOG
;;
yellow)
echo "cpqcpu.$2.rrd"
echo "$3:$4:$5:$6"
echo "cpqcpu.$2.rrd" >> $TMPLOG
echo "$3:$4:$5:$6" >> $TMPLOG
;;
red)
echo "cpqcpu.$2.rrd"
echo "$3:$4:$5:$6"
echo "cpqcpu.$2.rrd" >> $TMPLOG
echo "$3:$4:$5:$6" >> $TMPLOG
;;
*)
;;
esac
fi
done
fi
exit 0
On Wed, Oct 18, 2006 at 04:54:44PM -0400, Rob Munsch wrote:
I still do not quite get how the actual data gets to rrd. It seems to me that there is an isolated client script that sends data which determines the status color.. then a DIFFERENT server-side script is used to send the SAME DATA to RRD for graphing. ?!?! yesno?
Why isn't there a way to use the existing data that the client's sending for graphing... for CPU, MEM, etc. isn't that already happening? or have i failed another concept here.
Writing clear documentation is difficult when you've designed all of this and know how everything fits together. So blame any missing or misleading docs on me :-)
The first thing to get your head around is that Hobbit has a very clear separation between *collecting* data, *analyzing* data, and *presenting* data.
Collecting data is done on the client; that's your script running ipmitool (or whatever you use to collect data) and sends it to the Hobbit server.
When the Hobbit server - specifically, the "hobbitd" process that receives messages from the network - gets your data via a "status" message, it creates a column with the color your script decided upon. But it doesn't do any analysis of the data. Instead, it just shoves the raw message out through one or more of the Hobbit "channels", and lets the hobbitd_* worker modules do whatever they want to do with it. One of these worker modules is "hobbitd_rrd", which (as the name implies) handles updating of RRD files. It has to pick out the interesting numbers from the raw status message, and put them into an RRD file. hobbitd_rrd knows how to handle several different kinds of status messages (cpu, disk, memory, network response times ...), but for custom tests that you write yourself, you'll have to provide the code to pick out those numbers you want to put into the graph. So this is where the second script comes into play: This script is the one with the "echo" statements that you cannot quite figure out how should work. Basically, whenever hobbitd_rrd sees a status message which is one of those listed in the "--extra-tests" option, then it hands over the parsing of the status message data to a script you've written. This script must then return the data that should go into the RRD file.
So this is the *analyzing* part, and it runs on the Hobbit server. It is done by your second script, which gets invoked by the hobbitd_rrd process.
The final part is the *presentation* of the data. Hobbit knows how to present the status message itself, so the only thing that is missing is the graph that should be created based on the data in the RRD file. For this, you must first create a definition in hobbitgraph.cfg, describing what RRD file your graph uses, which datasets should go on the graph, whether they are lines or stacked areas etc. Next, you must associate the graph definition with the name of your status column; to do that, you add it to the TEST2RRD definition in hobbitserver.cfg. So if your column name is "ipmi" and the graph definition is also "ipmi", then you just add "ipmi" to the TEST2RRD setting. If the columnname and the graph definition differs, you add a COLUMNNAME=GRAPHNAME to TEST2RRD. Last, you must add the graph to the GRAPHS definition (yes, this is messy - I know) so it will show up also on the "trends" status. And here you use the filename of the RRD file (without ".rrd").
This is quite a bit of work you need to do. It's flexible, but it takes some time to get used to. There *is* a simpler mechanism called the "ncv" graphs, but this only works if the data in your status message is formatted as name-colon-value pairs which then go directly into an RRD file.
I hope this helps.
Regards, Henrik
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
You are trying to combine two scripts into one. Read the FAQ.
The faq, of note, has some contradictions, while we're on the subject. Frex:
"If you want the graph included with the other graphs on the trends column, you must add it to the GRAPHS setting in the ~hobbit/server/etc/hobbitserver.cfg file.
GRAPHS="la,disk,<... lots more ...>,bbproxy,hobbitd,slab"Okay. Did that with cputemp. Then at the bottom:
"Make sure you have ncv listed in the GRAPHS setting in hobbitserver.cfg. (Don't ask why - just take my word that it must be there)."
Huh? Does this refer to TEST2RRD and is a typo? Does it mean the "extra setting" vaguely described as
So we add an extra setting to hobbitserver.cfg:
NCV_slab="inodecache:GAUGE,dentrycache:GAUGE"which i kept after the existing statements in the Graph section, for lack of any more specific notes.
Or does it mean that "ncv" must now appear in GRAPHS= by itself, and this is the first mention of it?
I have spent all day working with this. Please don't respond to threads with "read the FAQ" when i quite obviously stated i have done so, and they are definitely not written for the novice in any event. It means either you have not read what I have written, or that you feel I am in some way not worth your time. This is very frustrating.
Thank you.
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 2:13 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block, Well i'm stuck already. What file is that block in? I can't find it
in 'server.cfg nor 'graph.cfg... It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all? That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
"Sir, I have no data."
Ok, so i did this
[cputemp] TITLE CPU Temp YAXIS Degrees Fahrenheit DEF:avg=cputemp.rrd:cputemp:AVERAGE AREA:cputemp#00CC00:CPU Temp Average -u 1.0 GPRINT:cputemp:LAST: \: %5.11f (cur) GPRINT:cputemp:MAX: \: %5.1lf (max) GPRINT:cputemp:MIN: \: %5.1lf (min) GPRINT:cputemp:AVERAGE: \: %5.1lf (avg)\n
and this
GRAPHS="la,disk,inode,qtree,memory,users,cputemp,.....8<...
as well, on the server. but what i get is a text link called
Status unchanged in 1 hours, 41 minutes Status message received from 10.10.10.12 hobbit graph cputemp
that leads to nothing. Not even a "nan" graph :P
I think maybe by echo is in the wrong place...? Should it be before or after the $BB msg command? Does it matter?
hobbit at currant:~/client/ext$ cat cputemp.sh #!/bin/sh
umask 133
COLUMN=cputemp COLOR=green MSG="Current CPU Temperature: " host=
hostnameipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=
tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Bad, or even Very Bad.
if $temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot! "else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain. "fi
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd" echo "$temp"
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR
date${MSG} "
exit 0
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNocyBvBcJFK6xYURAm1xAJ0eMz7qu+x+i1UZOXH0365rY1DefgCfT5P0 bh0QNDTExMNmwD3tyNpOkZc= =q/sU -----END PGP SIGNATURE-----
A note about NCV -- this is Yet Another Way to snag numbers and put them into graphs, but I never got it to work. So I did it the hard way. Others have gotten NCV to work. The problem is that NCV instructions are unnecessary if you write your own parser.
I agree that the documentation is not written as a tutorial, but more as a dump of working code.
GLH
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 2:58 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hubbard, Greg L wrote:
Rob,
You are trying to combine two scripts into one. Read the FAQ.
The faq, of note, has some contradictions, while we're on the subject. Frex:
"If you want the graph included with the other graphs on the trends column, you must add it to the GRAPHS setting in the ~hobbit/server/etc/hobbitserver.cfg file.
GRAPHS="la,disk,<... lots more ...>,bbproxy,hobbitd,slab"Okay. Did that with cputemp. Then at the bottom:
"Make sure you have ncv listed in the GRAPHS setting in hobbitserver.cfg. (Don't ask why - just take my word that it must be there)."
Huh? Does this refer to TEST2RRD and is a typo? Does it mean the "extra setting" vaguely described as
So we add an extra setting to hobbitserver.cfg:
NCV_slab="inodecache:GAUGE,dentrycache:GAUGE"which i kept after the existing statements in the Graph section, for lack of any more specific notes.
Or does it mean that "ncv" must now appear in GRAPHS= by itself, and this is the first mention of it?
I have spent all day working with this. Please don't respond to threads with "read the FAQ" when i quite obviously stated i have done so, and they are definitely not written for the novice in any event. It means either you have not read what I have written, or that you feel I am in some way not worth your time. This is very frustrating.
Thank you.
-----Original Message----- From: Rob Munsch [mailto:rmunsch at solutionsforprogress.com] Sent: Wednesday, October 18, 2006 2:13 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Sending data from script to an rrd
Ralph Mitchell wrote:
On 10/18/06, Rob Munsch <rmunsch at solutionsforprogress.com> wrote:
When you add the --extra-script option in the [rrdstatus] block, Well i'm stuck already. What file is that block in? I can't find it
in 'server.cfg nor 'graph.cfg... It's in server/etc/hobbitlaunch.cfg
yeah found that with a grep rrdstatus, heh oops.
So something like
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd"
probably isn't far off after all? That should do it, yes. That's the format for the RRD creation, and the name to create. If you just want to push out one value, you'd follow that with:
echo "$temp"
I seem to still be doing something wrong...
I don't think hobbit_rrd will recreate the rrd if it already exists, so you'd have to delete it sometime to get it rebuilt with the extra variables. Wave bye-bye to your old data...
"Sir, I have no data."
Ok, so i did this
[cputemp] TITLE CPU Temp YAXIS Degrees Fahrenheit DEF:avg=cputemp.rrd:cputemp:AVERAGE AREA:cputemp#00CC00:CPU Temp Average -u 1.0 GPRINT:cputemp:LAST: \: %5.11f (cur) GPRINT:cputemp:MAX: \: %5.1lf (max) GPRINT:cputemp:MIN: \: %5.1lf (min) GPRINT:cputemp:AVERAGE: \: %5.1lf (avg)\n
and this
GRAPHS="la,disk,inode,qtree,memory,users,cputemp,.....8<...
as well, on the server. but what i get is a text link called
Status unchanged in 1 hours, 41 minutes Status message received from 10.10.10.12 hobbit graph cputemp
that leads to nothing. Not even a "nan" graph :P
I think maybe by echo is in the wrong place...? Should it be before or after the $BB msg command? Does it matter?
hobbit at currant:~/client/ext$ cat cputemp.sh #!/bin/sh
umask 133
COLUMN=cputemp COLOR=green MSG="Current CPU Temperature: " host=
hostnameipmitool sensor | grep Temp | awk -F\| '{print $1,$2,$3,$4}' > $BBTMP/cputemp.$host
cpu_temp=
tail -n 4 $BBTMP/cputemp.$host |head -n 1 | awk '{print $2}' |sed 's/\..*//g'Converting to Fahrenheit.
temp=$((${cpu_temp}*9/5+32))
Ok, we have the temp in degrees as an integer.
Now to see if it's Bad, or even Very Bad.
if $temp > 130 then COLOR=red MSG="${MSG} $temp degrees Fahrenheit!
Hothothot! "else MSG="${MSG} $temp Degrees Fahrenheit
Chillin' like a Villain. "fi
echo "DS:cputemp:GAUGE:600:0:U" echo "cputemp.rrd" echo "$temp"
Get this to the hobbit server
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR
date${MSG} "
exit 0
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNocyBvBcJFK6xYURAm1xAJ0eMz7qu+x+i1UZOXH0365rY1DefgCfT5P0 bh0QNDTExMNmwD3tyNpOkZc= =q/sU -----END PGP SIGNATURE-----
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Wed, Oct 18, 2006 at 03:43:40PM -0500, Hubbard, Greg L wrote:
I agree that the documentation is not written as a tutorial, but more as a dump of working code.
If someone does write a tutorial, please send me a copy of it so I can include it with the Hobbit docs. As I said, it is really hard to forget everything you know and try and describe the proces to someone who has no prior knowledge about how it works.
Regards, Henrik
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Henrik Stoerner wrote:
On Wed, Oct 18, 2006 at 03:43:40PM -0500, Hubbard, Greg L wrote:
I agree that the documentation is not written as a tutorial, but more as a dump of working code.
If someone does write a tutorial, please send me a copy of it so I can include it with the Hobbit docs. As I said, it is really hard to forget everything you know and try and describe the proces to someone who has no prior knowledge about how it works.
I've nothing to forget! ,-) As soon as I get this working i'll try to cobble it all into a working step-by-step. it might be a few days...
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNp/gBvBcJFK6xYURAobNAJ9ml20FYgvyHv34t6pakHFUiGq3ogCeOFIo +1w6wCxgwR4eTMHDnKjkmUA= =7r1Q -----END PGP SIGNATURE-----
participants (5)
-
greg.hubbard＠eds.com
-
henrik＠hswn.dk
-
joel＠carnat.net
-
ralphmitchell＠gmail.com
-
rmunsch＠solutionsforprogress.com