Graphing issue with data that only changes once every 24 hours
Hi All,
I've been working on a VS backup solution which I have implemented using proftpd with mySQL, ftpmail (perl script) and quotas on Debian. Essentially whenever a client uploads data, ftpmail reads the transfer log from a fifo, and I've edited the perl script to send status messages through to Xymon. The solution is provided as only being used for backups and the client software sends in its own logs to our Xymon server. This is a differential backup and the client software handles the deletion of aged backup files.
All is working robustly on this front and in Xymon I collect the following data for graphing:
usage : 83.09 allowance : 100.00
These figures are currently sent in MB (I need to write a percentile calculation), but only once every 24 hours. This all falls over when it comes to trying to graph the data in Xymon as it is not a constant feed of data every 300 seconds like other tests. From the rrdtool documentation it seems like the data I am presenting is unsuitable for graphing in the normal method. I've tried just about every data type and I only get a line drawn when data is received (else the graph drops out on every step without data).
I realise I could run a client test on the VS that runs du on each chrooted FTP folder every 5 minutes, but this seems like an unnecessary load when the data changes with the 24 hour frequency and it scales horribly. Also throwing out ~100 status messages for ~100 different clients every few minutes to one server (say if I was to only run the test every 24 hours and repeat the data sent every 5 minutes) may be a little unreliable. Possibly changing the xymon client on the VS to a server and running everything locally would be the way to go; but still load wise this isn't great when compared to the methodology of only sending data on upload - though functional I guess.
Is there a way of providing a custom step in rrdtool for a single test from Xymon? Has anyone got any other ideas of how I might graph this? Am I missing something obvious?
Cheers, Phil
In <AANLkTi=OXLO7cVThaCK8vmZNNaumTGXfwTf+oPxuhqAh at mail.gmail.com> Phil Meech <pmeech at gmail.com> writes:
These figures are currently sent in MB (I need to write a percentile calculation), but only once every 24 hours. This all falls over when it comes to trying to graph the data in Xymon as it is not a constant feed of data every 300 seconds like other tests. From the rrdtool documentation it seems like the data I am presenting is unsuitable for graphing in the normal method. I've tried just about every data type and I only get a line drawn when data is received (else the graph drops out on every step without data).
When the RRD file gets created, one of the parameters is how often updates should be expected (it's called the "step" value). Xymon uses 300 seconds for this value. Another parameter is the "heartbeat" setting, which is the maximum time allowed between updates - if updates do not arrive within "heartbeat" seconds, then RRD won't show a graph because it believes the data is too sparse to make sense. Xymon uses a value of 600 seconds for the heartbeat (allowing us to miss one update and still keep the graph).
All of these settings are fixed when the RRD file is created. So what you need to do is re-create the RRD file with settings appropriate for your data. Something like this would probably work:
rrdtool create mydata.rrd
--step 86400
DS:mbytes:GAUGE:192800:0:U
RRA:AVERAGE:0.5:1:730
This create an RRD file where updates are expected once a day (86400 seconds = 24 hours). There is one set of data in it, called "mbytes" - this is a GAUGE, and it is minimum 0 and has no upper limit ("U" = undefined). There is only one RRA archive which holds 730 data-points, so you have room for two years of data.
You can use "rrdtool info" on your current rrd file to see what it looks like, in case you are not sure of some of these settings.
Once the RRD file exists, Xymon shouldn't have any problems with performing the updates or building graphs.
Regards, Henrik
participants (2)
-
henrik@hswn.dk
-
pmeech@gmail.com