On Wed, Oct 18, 2006 at 04:54:44PM -0400, Rob Munsch wrote:
I still do not quite get how the actual data gets to rrd. It seems to me that there is an isolated client script that sends data which determines the status color.. then a DIFFERENT server-side script is used to send the SAME DATA to RRD for graphing. ?!?! yesno?
Why isn't there a way to use the existing data that the client's sending for graphing... for CPU, MEM, etc. isn't that already happening? or have i failed another concept here.
Writing clear documentation is difficult when you've designed all of this and know how everything fits together. So blame any missing or misleading docs on me :-)
The first thing to get your head around is that Hobbit has a very clear separation between *collecting* data, *analyzing* data, and *presenting* data.
Collecting data is done on the client; that's your script running ipmitool (or whatever you use to collect data) and sends it to the Hobbit server.
When the Hobbit server - specifically, the "hobbitd" process that receives messages from the network - gets your data via a "status" message, it creates a column with the color your script decided upon. But it doesn't do any analysis of the data. Instead, it just shoves the raw message out through one or more of the Hobbit "channels", and lets the hobbitd_* worker modules do whatever they want to do with it. One of these worker modules is "hobbitd_rrd", which (as the name implies) handles updating of RRD files. It has to pick out the interesting numbers from the raw status message, and put them into an RRD file. hobbitd_rrd knows how to handle several different kinds of status messages (cpu, disk, memory, network response times ...), but for custom tests that you write yourself, you'll have to provide the code to pick out those numbers you want to put into the graph. So this is where the second script comes into play: This script is the one with the "echo" statements that you cannot quite figure out how should work. Basically, whenever hobbitd_rrd sees a status message which is one of those listed in the "--extra-tests" option, then it hands over the parsing of the status message data to a script you've written. This script must then return the data that should go into the RRD file.
So this is the *analyzing* part, and it runs on the Hobbit server. It is done by your second script, which gets invoked by the hobbitd_rrd process.
The final part is the *presentation* of the data. Hobbit knows how to present the status message itself, so the only thing that is missing is the graph that should be created based on the data in the RRD file. For this, you must first create a definition in hobbitgraph.cfg, describing what RRD file your graph uses, which datasets should go on the graph, whether they are lines or stacked areas etc. Next, you must associate the graph definition with the name of your status column; to do that, you add it to the TEST2RRD definition in hobbitserver.cfg. So if your column name is "ipmi" and the graph definition is also "ipmi", then you just add "ipmi" to the TEST2RRD setting. If the columnname and the graph definition differs, you add a COLUMNNAME=GRAPHNAME to TEST2RRD. Last, you must add the graph to the GRAPHS definition (yes, this is messy - I know) so it will show up also on the "trends" status. And here you use the filename of the RRD file (without ".rrd").
This is quite a bit of work you need to do. It's flexible, but it takes some time to get used to. There *is* a simpler mechanism called the "ncv" graphs, but this only works if the data in your status message is formatted as name-colon-value pairs which then go directly into an RRD file.
I hope this helps.
Regards, Henrik