I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
You could do that server-side. Look at the man page for xymond_rrd. The --extra-script option allows you to pipe incoming data through a script to process. You could take the report time and subtract 1 min, 2 min, etc, then emit rrd updates as required. The client would still send all 5 values at the same time, but they'd be recorded as if received over the preceding 5 minute block.
Ralph Mitchell On Aug 15, 2014 9:55 AM, "David Welker" <dewelker at gmail.com> wrote:
I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
We've taken to using Graphite for metrics. It's a one-liner to put values collected from xymon into graphite; it's also possible to alert on graphite values in xymon
(I confess I didn't write those scripts, but it's basically grabbing info from a URL designed for that purpose)
Our tools are based on these, written by Dewey Sasser https://github.com/dmsasser/graphiteutils
This sounded like a great idea, so I got the status showing up via the extra-scripts option, but the info on the data message is a little vague. According to the example, I just echo the DS statements, then the dataset name on a line by itself, followed by the data for one dataset on a line by itself, which doesn't seem to do anything in the way of creating any of the multiple rrds I need, nor does it address the time consideration for multple values (I need to insert 5 values a minute apart from 5 mins. before putting them into the rrd - I know this can be done with the rrdtool update command, but can it be done here somehow?) Anybody have an example of a multi-dataset case being run via the extra-scripts option of the xymond_rrd they'd be willing to share?
Thanks! David
On Fri, Aug 15, 2014 at 10:09 AM, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
You could do that server-side. Look at the man page for xymond_rrd. The --extra-script option allows you to pipe incoming data through a script to process. You could take the report time and subtract 1 min, 2 min, etc, then emit rrd updates as required. The client would still send all 5 values at the same time, but they'd be recorded as if received over the preceding 5 minute block.
Ralph Mitchell On Aug 15, 2014 9:55 AM, "David Welker" <dewelker at gmail.com> wrote:
I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Ralph's suggestion looks very interesting, and means you can have it apply to any and all clients without having to make changes to each one.
In case you can't get it working for some reason, I think something client-side like the following might work:
#!/bin/sh
die() { echo "$@" >&2; exit 1; }
[ "$XYMON" -a "$XYMONSRV" -a "$MACHINE" ] || die "Please run within a Xymon
environment"
VALUES=get_values # gets 5 numbers
set - ""$SAMPLES
[ $# -eq 5 ] || die "Value count $# was not 5: $VALUES"
SLEEP=0
DELAY=60 # 300/5+1
for VALUE in $VALUES; do
MSG=printf "data $MACHINE.trends\n[filename.rrd]\nDS:dsname:$DELAY:U:U $VALUE\n"
( sleep $SLEEP; echo "$MSG" | $XYMON $XYMSRV @ ) &
let SLEEP=$SLEEP+$DELAY
done
This should fire off 5 background tasks, each waiting a multiple of 60 seconds before sending in their value to the RRD collector, thus staggering the updates across the next 5 minutes.
J
On 20 August 2014 01:22, David Welker <dewelker at gmail.com> wrote:
This sounded like a great idea, so I got the status showing up via the extra-scripts option, but the info on the data message is a little vague. According to the example, I just echo the DS statements, then the dataset name on a line by itself, followed by the data for one dataset on a line by itself, which doesn't seem to do anything in the way of creating any of the multiple rrds I need, nor does it address the time consideration for multple values (I need to insert 5 values a minute apart from 5 mins. before putting them into the rrd - I know this can be done with the rrdtool update command, but can it be done here somehow?) Anybody have an example of a multi-dataset case being run via the extra-scripts option of the xymond_rrd they'd be willing to share?
Thanks! David
On Fri, Aug 15, 2014 at 10:09 AM, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
You could do that server-side. Look at the man page for xymond_rrd. The --extra-script option allows you to pipe incoming data through a script to process. You could take the report time and subtract 1 min, 2 min, etc, then emit rrd updates as required. The client would still send all 5 values at the same time, but they'd be recorded as if received over the preceding 5 minute block.
Ralph Mitchell On Aug 15, 2014 9:55 AM, "David Welker" <dewelker at gmail.com> wrote:
I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Now that I've had a chance to read the man page, I see I wasn't remembering it properly. I thought the last bit:
# The data
echo "$TEMP:$WIND:$HMTY:$RAIN"included the timestamp for the RRD insert. So, here's another approach that may work. Instead of echoing out the data, it could be directly written to the rrd, like this:
RRD=/home/xymon/data/rrd/<hostname>/thing.rrd
TIME=`date +%s`
MINUS1=`expr $TIME - 60`
MINUS2=`expr $MINUS1 - 60`
MINUS3=`expr $MINUS2 - 60`
MINUS4=`expr $MINUS3 - 60`
rrdtool update $RRD $MINUS4:$DATA4
rrdtool update $RRD $MINUS3:$DATA3
rrdtool update $RRD $MINUS2:$DATA2
rrdtool update $RRD $MINUS1:$DATA1
rrdtool update $RRD $TIME:$DATA0You would need to break out the 5 data points from the original message as DATA0 through DATA4. As long as you don't echo out the data values, I *think* xymond_rrd won't update the rrd for itself. Even it it does enter a NaN value, you've just fired off 5 real data points, which ought to be more significant that the NaN. By updating the Xymon rrd for the server, you still get the graphs in the trends column, with multiple time spreads, zooming, etc.
I hope that makes sense. I don't have a handy test server to try it out on.
Another possibility, if you (or someone nearby) are handy with C. The xymond_sample.c program is included in the Xymon source. It demonstrates how to hook into the message flow and process incoming data. It ought to be possible to achieve the above rrd updates by receiving the message and dropping the data points back into the message stream at 60 second intervals. Maybe. You *really* need a spare test server to try that out... :-)
Ralph Mitchell
On Tue, Aug 19, 2014 at 11:22 AM, David Welker <dewelker at gmail.com> wrote:
This sounded like a great idea, so I got the status showing up via the extra-scripts option, but the info on the data message is a little vague. According to the example, I just echo the DS statements, then the dataset name on a line by itself, followed by the data for one dataset on a line by itself, which doesn't seem to do anything in the way of creating any of the multiple rrds I need, nor does it address the time consideration for multple values (I need to insert 5 values a minute apart from 5 mins. before putting them into the rrd - I know this can be done with the rrdtool update command, but can it be done here somehow?) Anybody have an example of a multi-dataset case being run via the extra-scripts option of the xymond_rrd they'd be willing to share?
Thanks! David
On Fri, Aug 15, 2014 at 10:09 AM, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
You could do that server-side. Look at the man page for xymond_rrd. The --extra-script option allows you to pipe incoming data through a script to process. You could take the report time and subtract 1 min, 2 min, etc, then emit rrd updates as required. The client would still send all 5 values at the same time, but they'd be recorded as if received over the preceding 5 minute block.
Ralph Mitchell On Aug 15, 2014 9:55 AM, "David Welker" <dewelker at gmail.com> wrote:
I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Right on target! Just before I sent that last response, I figured out Xymon wouldn't let me use the N-300, etc. notations and decided to try it via Rrdtool directly, so at least I was thinking along the right lines! Only three problems...
- Minor problem - just an fyi, but trying to update a non-existent rrd fails, so you have to add a bit more code to create.one first...no big deal, but brings me to....
2.) If I want to eventually show all 5 figures, plus an average, min, max, and last, how do I... a.) define the rrd? I have: DS:time:gauge:600:U:U RRA:AVERAGE:0.5:1:1200 RRA:MIN:0.5:24:2400 RRA:MAX:0.5:24:2400 RRA:AVERAGE:0.5:24:2400 b.) define the graph? I have: [tvalues] TITLE Time Values YAXIS Time (m/s) DEF:t1=time.rrd:t1:AVERAGE DEF:t2=time.rrd:t2:AVERAGE DEF:t3=time.rrd:t3:AVERAGE DEF:t4=time.rrd:t4:AVERAGE DEF:t5=time.rrd:t5:AVERAGE LINE2: GPRINT:t1:%6.11f%s (cur) GPRINT:t2::\: %6.11f%s (cur) GPRINT:t3::\: %6.11f%s (cur) GPRINT:t4::\: %6.11f%s (cur) GPRINT:t5::\: %6.11f%s (cur) GPRINT:t6: \: %6.11f%s (max) GPRINT:t6: \: %6.11f%s (min) GPRINT:t6: \: %6.11f%s (avg)
- Note: Assuming that defining the RRAs was correct, I really want the RRA values for t6
3.) lastly, I set up the client-side script to collect the data via a clientlaunch.cfg task, the clientlaunch xymond_rrd status then runs my script to create the rrds (via the extra-script/extra-tests), so do I need extra-script/extra-tests on the xymond_rrd data line?
Currently, both the rrds and the graph are being created, but empty.
Thanks for your suggestions!
Any and all help appreciated,
David
On Tue, Aug 19, 2014 at 11:29 PM, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
Now that I've had a chance to read the man page, I see I wasn't remembering it properly. I thought the last bit:
# The data echo "$TEMP:$WIND:$HMTY:$RAIN"included the timestamp for the RRD insert. So, here's another approach that may work. Instead of echoing out the data, it could be directly written to the rrd, like this:
RRD=/home/xymon/data/rrd/<hostname>/thing.rrd TIME=`date +%s` MINUS1=`expr $TIME - 60` MINUS2=`expr $MINUS1 - 60` MINUS3=`expr $MINUS2 - 60` MINUS4=`expr $MINUS3 - 60` rrdtool update $RRD $MINUS4:$DATA4 rrdtool update $RRD $MINUS3:$DATA3 rrdtool update $RRD $MINUS2:$DATA2 rrdtool update $RRD $MINUS1:$DATA1 rrdtool update $RRD $TIME:$DATA0You would need to break out the 5 data points from the original message as DATA0 through DATA4. As long as you don't echo out the data values, I *think* xymond_rrd won't update the rrd for itself. Even it it does enter a NaN value, you've just fired off 5 real data points, which ought to be more significant that the NaN. By updating the Xymon rrd for the server, you still get the graphs in the trends column, with multiple time spreads, zooming, etc.
I hope that makes sense. I don't have a handy test server to try it out on.
Another possibility, if you (or someone nearby) are handy with C. The xymond_sample.c program is included in the Xymon source. It demonstrates how to hook into the message flow and process incoming data. It ought to be possible to achieve the above rrd updates by receiving the message and dropping the data points back into the message stream at 60 second intervals. Maybe. You *really* need a spare test server to try that out... :-)
Ralph Mitchell
On Tue, Aug 19, 2014 at 11:22 AM, David Welker <dewelker at gmail.com> wrote:
This sounded like a great idea, so I got the status showing up via the extra-scripts option, but the info on the data message is a little vague. According to the example, I just echo the DS statements, then the dataset name on a line by itself, followed by the data for one dataset on a line by itself, which doesn't seem to do anything in the way of creating any of the multiple rrds I need, nor does it address the time consideration for multple values (I need to insert 5 values a minute apart from 5 mins. before putting them into the rrd - I know this can be done with the rrdtool update command, but can it be done here somehow?) Anybody have an example of a multi-dataset case being run via the extra-scripts option of the xymond_rrd they'd be willing to share?
Thanks! David
On Fri, Aug 15, 2014 at 10:09 AM, Ralph Mitchell <ralphmitchell at gmail.com
wrote:
You could do that server-side. Look at the man page for xymond_rrd. The --extra-script option allows you to pipe incoming data through a script to process. You could take the report time and subtract 1 min, 2 min, etc, then emit rrd updates as required. The client would still send all 5 values at the same time, but they'd be recorded as if received over the preceding 5 minute block.
Ralph Mitchell On Aug 15, 2014 9:55 AM, "David Welker" <dewelker at gmail.com> wrote:
I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Ralph and Jeremy,
I was able to get a graph, but the 5 figures for one column are in the k/seconds range while the other are milliseconds, so while one shows up rather well, the other hovers around the 0 line. 1.) Is there a way to change this - like the default time of a Xymon graph, say last 2 hours, on a column status page? 2.) What might be even better, is there a way to display 2 graphs on a column status page with each in their own graph? 3.) Also, is it possible to display the actual data point values captured, on the GPRINT line as I mentioned earlier?
Thanks again! David
On Thu, Aug 21, 2014 at 11:06 AM, David Welker <dewelker at gmail.com> wrote:
Right on target! Just before I sent that last response, I figured out Xymon wouldn't let me use the N-300, etc. notations and decided to try it via Rrdtool directly, so at least I was thinking along the right lines! Only three problems...
- Minor problem - just an fyi, but trying to update a non-existent rrd fails, so you have to add a bit more code to create.one first...no big deal, but brings me to....
2.) If I want to eventually show all 5 figures, plus an average, min, max, and last, how do I... a.) define the rrd? I have: DS:time:gauge:600:U:U RRA:AVERAGE:0.5:1:1200 RRA:MIN:0.5:24:2400 RRA:MAX:0.5:24:2400 RRA:AVERAGE:0.5:24:2400 b.) define the graph? I have: [tvalues] TITLE Time Values YAXIS Time (m/s) DEF:t1=time.rrd:t1:AVERAGE DEF:t2=time.rrd:t2:AVERAGE DEF:t3=time.rrd:t3:AVERAGE DEF:t4=time.rrd:t4:AVERAGE DEF:t5=time.rrd:t5:AVERAGE LINE2: GPRINT:t1:%6.11f%s (cur) GPRINT:t2::\: %6.11f%s (cur) GPRINT:t3::\: %6.11f%s (cur) GPRINT:t4::\: %6.11f%s (cur) GPRINT:t5::\: %6.11f%s (cur) GPRINT:t6: \: %6.11f%s (max) GPRINT:t6: \: %6.11f%s (min) GPRINT:t6: \: %6.11f%s (avg)
- Note: Assuming that defining the RRAs was correct, I really want the RRA values for t6
3.) lastly, I set up the client-side script to collect the data via a clientlaunch.cfg task, the clientlaunch xymond_rrd status then runs my script to create the rrds (via the extra-script/extra-tests), so do I need extra-script/extra-tests on the xymond_rrd data line?
Currently, both the rrds and the graph are being created, but empty.
Thanks for your suggestions!
Any and all help appreciated,
David
On Tue, Aug 19, 2014 at 11:29 PM, Ralph Mitchell <ralphmitchell at gmail.com> wrote:
Now that I've had a chance to read the man page, I see I wasn't remembering it properly. I thought the last bit:
# The data echo "$TEMP:$WIND:$HMTY:$RAIN"included the timestamp for the RRD insert. So, here's another approach that may work. Instead of echoing out the data, it could be directly written to the rrd, like this:
RRD=/home/xymon/data/rrd/<hostname>/thing.rrd TIME=`date +%s` MINUS1=`expr $TIME - 60` MINUS2=`expr $MINUS1 - 60` MINUS3=`expr $MINUS2 - 60` MINUS4=`expr $MINUS3 - 60` rrdtool update $RRD $MINUS4:$DATA4 rrdtool update $RRD $MINUS3:$DATA3 rrdtool update $RRD $MINUS2:$DATA2 rrdtool update $RRD $MINUS1:$DATA1 rrdtool update $RRD $TIME:$DATA0You would need to break out the 5 data points from the original message as DATA0 through DATA4. As long as you don't echo out the data values, I *think* xymond_rrd won't update the rrd for itself. Even it it does enter a NaN value, you've just fired off 5 real data points, which ought to be more significant that the NaN. By updating the Xymon rrd for the server, you still get the graphs in the trends column, with multiple time spreads, zooming, etc.
I hope that makes sense. I don't have a handy test server to try it out on.
Another possibility, if you (or someone nearby) are handy with C. The xymond_sample.c program is included in the Xymon source. It demonstrates how to hook into the message flow and process incoming data. It ought to be possible to achieve the above rrd updates by receiving the message and dropping the data points back into the message stream at 60 second intervals. Maybe. You *really* need a spare test server to try that out... :-)
Ralph Mitchell
On Tue, Aug 19, 2014 at 11:22 AM, David Welker <dewelker at gmail.com> wrote:
This sounded like a great idea, so I got the status showing up via the extra-scripts option, but the info on the data message is a little vague. According to the example, I just echo the DS statements, then the dataset name on a line by itself, followed by the data for one dataset on a line by itself, which doesn't seem to do anything in the way of creating any of the multiple rrds I need, nor does it address the time consideration for multple values (I need to insert 5 values a minute apart from 5 mins. before putting them into the rrd - I know this can be done with the rrdtool update command, but can it be done here somehow?) Anybody have an example of a multi-dataset case being run via the extra-scripts option of the xymond_rrd they'd be willing to share?
Thanks! David
On Fri, Aug 15, 2014 at 10:09 AM, Ralph Mitchell < ralphmitchell at gmail.com> wrote:
You could do that server-side. Look at the man page for xymond_rrd. The --extra-script option allows you to pipe incoming data through a script to process. You could take the report time and subtract 1 min, 2 min, etc, then emit rrd updates as required. The client would still send all 5 values at the same time, but they'd be recorded as if received over the preceding 5 minute block.
Ralph Mitchell On Aug 15, 2014 9:55 AM, "David Welker" <dewelker at gmail.com> wrote:
I've been looking, but haven't seen anything in the documentation, on the website, or in the mailing list that addresses this:
I need to monitor 5 values that are received all at the same time. I'd like to insert the values at one minute intervals getting a consolidated data point (CDP) every 5 minutes that I can graph.
For example, I ping a host via client script, 5 times, returning all of the data at one time. I want to put all 5 values in the database (rrd), but spaced apart every minute so I get a decent CDP. I could probably do this if this was on the server (could I?), but the script HAS to be on the client in this case. Is this even doable?
Thanks, in advance, for any suggestions!
David
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
On 28 August 2014 00:15, David Welker <dewelker at gmail.com> wrote:
I was able to get a graph, but the 5 figures for one column are in the k/seconds range while the other are milliseconds, so while one shows up rather well, the other hovers around the 0 line. 1.) Is there a way to change this - like the default time of a Xymon graph, say last 2 hours, on a column status page?
Sorry, I'm not quite sure what you mean. If you want each data source graphed on its own graph, rather than all in one, then I don't think it can be done out-of-the-box. However, multi-line graphs can be tweaked to display different numbers of sources on the one graph. If you right-click on the graph and show the image in its own window, you can then adjust the URL parameters "first" and "count". So there's a way to get "showgraph" to display just the one line on its own, but I don't think this can be adjusted for the column status page without changing the source code.
2.) What might be even better, is there a way to display 2 graphs on a column status page with each in their own graph?
My "disk" status page shows two graphs, each with four filesystems. Some smarts in the multi-graph processing will detect if the count of data sources exceeds a threshold (I think it's 6) and then splits the display into two or more graphs.
There's also some special processing for Devmon-derived status pages, such that a comment like this will adjust the number of separate graphs displayed:
<!-- linecount=9 -->
I'm pretty sure I've (ab)used this in the past, to take control over a graph on a status page, even though Devmon was not the source of the status or data.
3.) Also, is it possible to display the actual data point values captured,
on the GPRINT line as I mentioned earlier?
I'd have thought GPRINT displays the values captured, until consolidated of course. It could be that you need to zoom in on the graph to avoid the consolidation when displaying.
J
participants (4)
-
betsy.schwartz＠gmail.com
-
dewelker＠gmail.com
-
jlaidman＠rebel-it.com.au
-
ralphmitchell＠gmail.com