combine status and data in same column with graph
Hi All,
I sent some text to status channel and some NCVs to the data channel for same test/columnname.
So on the same script I have the followings
...
$XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR date
${MSG}
"$XYMON $XYMSRV "data $MACHINE.$COLUMN
foo: 1 bar: 2 "
Then I placed the columnname=ncv in TEST2RRD and restarted xymon
rrdtool dump $COLUMN.rrd shows all the text from status msgs in it, yikes!
How do I tell xymon to look at only the data channel for the rrds and leave the status channel alone for this test/columnname?
Thanks
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Asif
It might be an either/or, situation otherwise you get the situation you've described, where both status_rrd and data_rrd channels try to update the RRD file.
You might find that adjusting the TEST2RRD variable to include only "columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
You can include the NCV in your status message, perhaps hidden inside a comment like this:
$XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR date
${MSG}<!-- foo: 1 bar: 2 --> "
But you'd also need to reformat your MSG to not contain anything that looks like NCV. It might be sufficient to remove all the colons and equals signs, for example.
If you want to send the data separately, perhaps use a trends message, like so:
$XYMON $XYMSRV "data $MACHINE.trends [columnname.rrd] DS:foo:GAUGE:600:0:U 1 DS:bar:GAUGE:600:0:U 2 "
Cheers Jeremy
On 13 November 2013 09:20, Asif Iqbal <vadud3 at gmail.com> wrote:
Hi All,
I sent some text to status channel and some NCVs to the data channel for same test/columnname.
So on the same script I have the followings
... $XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR
date${MSG} "$XYMON $XYMSRV "data $MACHINE.$COLUMN
foo: 1 bar: 2 "
Then I placed the columnname=ncv in TEST2RRD and restarted xymon
rrdtool dump $COLUMN.rrd shows all the text from status msgs in it, yikes!
How do I tell xymon to look at only the data channel for the rrds and leave the status channel alone for this test/columnname?
Thanks
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
On Tue, Nov 12, 2013 at 7:21 PM, Jeremy Laidman <jlaidman at rebel-it.com.au>wrote:
Asif
It might be an either/or, situation otherwise you get the situation you've described, where both status_rrd and data_rrd channels try to update the RRD file.
You might find that adjusting the TEST2RRD variable to include only "columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
Looks like if I do not have columnname=ncv it does not generate a rrd file.
You can include the NCV in your status message, perhaps hidden inside a comment like this:
$XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR
date${MSG}<!-- foo: 1 bar: 2 --> "
But you'd also need to reformat your MSG to not contain anything that looks like NCV. It might be sufficient to remove all the colons and equals signs, for example.
I did some clean up so nothing in MSG has NCV like data and I added the NCV data in the <-- ... --> . It did create the rrd file with the correct NCV keys, but values are zeros. It is dropping the values for some reason? So I am almost there if I can figure out why the values are not showing up.
The other option would be using --extra-test and --extra-script and parse the values from the status channel to generate rrd?
If you want to send the data separately, perhaps use a trends message, like so:
I want the graph show up in the test page, so sending it to just trends won't work, right?
$XYMON $XYMSRV "data $MACHINE.trends [columnname.rrd] DS:foo:GAUGE:600:0:U 1 DS:bar:GAUGE:600:0:U 2 "
Cheers Jeremy
On 13 November 2013 09:20, Asif Iqbal <vadud3 at gmail.com> wrote:
Hi All,
I sent some text to status channel and some NCVs to the data channel for same test/columnname.
So on the same script I have the followings
... $XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR
date${MSG} "$XYMON $XYMSRV "data $MACHINE.$COLUMN
foo: 1 bar: 2 "
Then I placed the columnname=ncv in TEST2RRD and restarted xymon
rrdtool dump $COLUMN.rrd shows all the text from status msgs in it, yikes!
How do I tell xymon to look at only the data channel for the rrds and leave the status channel alone for this test/columnname?
Thanks
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
On 14 November 2013 03:52, Asif Iqbal <vadud3 at gmail.com> wrote:
You might find that adjusting the TEST2RRD variable to include only
"columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
Looks like if I do not have columnname=ncv it does not generate a rrd file.
:-(
I did some clean up so nothing in MSG has NCV like data and I added the NCV data in the <-- ... --> . It did create the rrd file with the correct NCV keys, but values are zeros. It is dropping the values for some reason? So I am almost there if I can figure out why the values are not showing up.
Are the values you're reporting counter-type (that is, always incrementing) or gauge-type? If you're sending gauge data and it's being treated as counter data, then you will often get mostly zeroes or NaN. By default, NCV data is expected to be DERIVE which is a type of counter. If you want to change to GAUGE, then you need to set NCV_columnname to describe this:
NCV_columnname="foo:GAUGE,bar:GAUGE"
The other option would be using --extra-test and --extra-script and parse
the values from the status channel to generate rrd?
Perhaps. These seem a lot trickier to use.
One thought I had about this was to setup "--extra-test" and "--extra-script" on the status channel, to filter out the status messages from xymond_rrd. If you use "--extra-script=/bin/true" and "--extra-test=columnname" then I think it will feed the data to "/bin/true" instead of parsing it itself. Of course "/bin/true" will produce no output, and hence no data.
I want the graph show up in the test page, so sending it to just trends won't work, right?
Right. It needs to be listed in TEST2RRD to show up in the test page.
Hmm, just noticed this in the doco:
"No graph on the status page, but OK on the trends page Make sure you have *ncv* listed in the GRAPHS setting in xymonserver.cfg. (Don't ask why - just take my word that it must be there)."
Do you have "ncv" listed in GRAPHS?
J
Hi Asif
Have a look here. https://wiki.xymonton.org/doku.php/monitors:db_cpu.ksh This is an example of what I think you are trying to do.
Regards Vernon
On 14 November 2013 08:39, Jeremy Laidman <jlaidman at rebel-it.com.au> wrote:
On 14 November 2013 03:52, Asif Iqbal <vadud3 at gmail.com> wrote:
You might find that adjusting the TEST2RRD variable to include only
"columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
Looks like if I do not have columnname=ncv it does not generate a rrd file.
:-(
I did some clean up so nothing in MSG has NCV like data and I added the NCV data in the <-- ... --> . It did create the rrd file with the correct NCV keys, but values are zeros. It is dropping the values for some reason? So I am almost there if I can figure out why the values are not showing up.
Are the values you're reporting counter-type (that is, always incrementing) or gauge-type? If you're sending gauge data and it's being treated as counter data, then you will often get mostly zeroes or NaN. By default, NCV data is expected to be DERIVE which is a type of counter. If you want to change to GAUGE, then you need to set NCV_columnname to describe this:
NCV_columnname="foo:GAUGE,bar:GAUGE"
The other option would be using --extra-test and --extra-script and parse
the values from the status channel to generate rrd?
Perhaps. These seem a lot trickier to use.
One thought I had about this was to setup "--extra-test" and "--extra-script" on the status channel, to filter out the status messages from xymond_rrd. If you use "--extra-script=/bin/true" and "--extra-test=columnname" then I think it will feed the data to "/bin/true" instead of parsing it itself. Of course "/bin/true" will produce no output, and hence no data.
I want the graph show up in the test page, so sending it to just trends won't work, right?
Right. It needs to be listed in TEST2RRD to show up in the test page.
Hmm, just noticed this in the doco:
"No graph on the status page, but OK on the trends page Make sure you have *ncv* listed in the GRAPHS setting in xymonserver.cfg. (Don't ask why - just take my word that it must be there)."
Do you have "ncv" listed in GRAPHS?
J
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
On Wed, Nov 13, 2013 at 7:41 PM, Vernon Everett <everett.vernon at gmail.com>wrote:
Hi Asif
Have a look here. https://wiki.xymonton.org/doku.php/monitors:db_cpu.ksh This is an example of what I think you are trying to do.
Looking at your link. What is db-cpu::100 means in GRAPHS ? I did not see any thing in xymonserver.cfg man page to explain the ``:100'' after the columnname
So essentially you are taking advantage of both status and data channel in the same script like I started with.
I will revisit my initial approach, except I will make sure there is no NCV like data, like Jeremy suggested, when sending to status channel or it screw up the rrd with garbage.
Thanks for the link.
Regards
Vernon
On 14 November 2013 08:39, Jeremy Laidman <jlaidman at rebel-it.com.au>wrote:
On 14 November 2013 03:52, Asif Iqbal <vadud3 at gmail.com> wrote:
You might find that adjusting the TEST2RRD variable to include only
"columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
Looks like if I do not have columnname=ncv it does not generate a rrd file.
:-(
I did some clean up so nothing in MSG has NCV like data and I added the NCV data in the <-- ... --> . It did create the rrd file with the correct NCV keys, but values are zeros. It is dropping the values for some reason? So I am almost there if I can figure out why the values are not showing up.
Are the values you're reporting counter-type (that is, always incrementing) or gauge-type? If you're sending gauge data and it's being treated as counter data, then you will often get mostly zeroes or NaN. By default, NCV data is expected to be DERIVE which is a type of counter. If you want to change to GAUGE, then you need to set NCV_columnname to describe this:
NCV_columnname="foo:GAUGE,bar:GAUGE"
The other option would be using --extra-test and --extra-script and parse
the values from the status channel to generate rrd?
Perhaps. These seem a lot trickier to use.
One thought I had about this was to setup "--extra-test" and "--extra-script" on the status channel, to filter out the status messages from xymond_rrd. If you use "--extra-script=/bin/true" and "--extra-test=columnname" then I think it will feed the data to "/bin/true" instead of parsing it itself. Of course "/bin/true" will produce no output, and hence no data.
I want the graph show up in the test page, so sending it to just trends won't work, right?
Right. It needs to be listed in TEST2RRD to show up in the test page.
Hmm, just noticed this in the doco:
"No graph on the status page, but OK on the trends page Make sure you have *ncv* listed in the GRAPHS setting in xymonserver.cfg. (Don't ask why - just take my word that it must be there). "
Do you have "ncv" listed in GRAPHS?
J
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
On Thu, Nov 14, 2013 at 12:52 PM, Asif Iqbal <vadud3 at gmail.com> wrote:
On Wed, Nov 13, 2013 at 7:41 PM, Vernon Everett <everett.vernon at gmail.com>wrote:
Hi Asif
Have a look here. https://wiki.xymonton.org/doku.php/monitors:db_cpu.ksh This is an example of what I think you are trying to do.
Looking at your link. What is db-cpu::100 means in GRAPHS ? I did not see any thing in xymonserver.cfg man page to explain the ``:100'' after the columnname
So essentially you are taking advantage of both status and data channel in the same script like I started with.
I will revisit my initial approach, except I will make sure there is no NCV like data, like Jeremy suggested, when sending to status channel or it screw up the rrd with garbage.
OK so this time it worked!! Thanks a lot to both Jeremy and Vernon!
I used the usual foo=ncv for TEST2RRD to generate rrd, GRAPHS=foo for it show up in trends column and SPLITNCV_foo="*;GAUGE" to create separate rrd, foo,ds1.rrd and foo,ds2.rrd, for each dataset.
I sent MSG to status channel and *made sure* there is NCV type data in
there. I used the sed trick to get rid of
:'' and ='' in there.
And I sent NCV type data to the data channel. Used just one script like this
$XYMON $XYMSRV "status $MACHINE.$COLUMN $COLOR date
${MSG}
"$XYMON $XYMSRV "data $MACHINE.$COLUMN $(echo) .. a : 2 ... .. b : 3 ... $(echo) "
In the graphs.cfg I have
[foo] FNPATTERN foo,(.*).rrd TITLE foo - ds1 and ds2 YAXIS % -u 100 -l 0 DEF:p at RRDIDX@=@RRDFN@:lambda:AVERAGE # must use lambda if using SPLITNCV LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@ GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur) GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max) GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min) GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n
This creates only one graph. How do I create multiple graphs? one for foo,ds1.rrd and one for foo,ds2.rrd ?
Thanks again for all the help.
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
On 15 November 2013 07:05, Asif Iqbal <vadud3 at gmail.com> wrote:
This creates only one graph. How do I create multiple graphs? one for foo,ds1.rrd and one for foo,ds2.rrd ?
I think the "::100" trick is related to how many lines to put onto one mult-graph before splitting into separate graphs. This is the "linecount" value. If you use "::1" then you should get two graphs, each one 1 line. I've never found this very predictable, and when I try to manipulate it, I get broken graphs. But worth you giving it a try.
Another thing you can do is output a comment saying "<!-- linecount=nnn -->" in your status message. Xymon detects this and sets the linecount value for displaying graphs. This is used by Devmon to control the number of lines on a graph. If you set it to 1, then you'll probably get two separate graphs for two DSes.
Another option is to create a separate graph for each DS. So you could have:
[foo] FNPATTERN foo,(.*).rrd TITLE foo - ds1 and ds2 ...
[foo1] FNPATTERN foo,ds1.rrd TITLE foo - ds1 ...
[foo2] FNPATTERN foo,ds2.rrd TITLE foo - ds2 ...
Then in hosts.cfg, adjust TRENDS to have foo:foo|foo1|foo2 if you want all three graphs on the trends page. However, the status page will only show the [foo] graph.
J
As Jeremy said, the number after the ::, is the number of data sets (lines) per graph. In the case of my example, we are generating a stack graph, of all CPU usage, so it will only look useful if all the datasets are graphed. And we run multiple databases per host. 100 is a number I chose, because we are unlikely to exceed 100 databases.
However, if you set that to something sane, like 4 or 5, you will only see 4 or 5 lines per graph, and multiple graphs will appear if necessary.
Regards Vernon
On 15 November 2013 01:52, Asif Iqbal <vadud3 at gmail.com> wrote:
On Wed, Nov 13, 2013 at 7:41 PM, Vernon Everett <everett.vernon at gmail.com>wrote:
Hi Asif
Have a look here. https://wiki.xymonton.org/doku.php/monitors:db_cpu.ksh This is an example of what I think you are trying to do.
Looking at your link. What is db-cpu::100 means in GRAPHS ? I did not see any thing in xymonserver.cfg man page to explain the ``:100'' after the columnname
So essentially you are taking advantage of both status and data channel in the same script like I started with.
I will revisit my initial approach, except I will make sure there is no NCV like data, like Jeremy suggested, when sending to status channel or it screw up the rrd with garbage.
Thanks for the link.
Regards
Vernon
On 14 November 2013 08:39, Jeremy Laidman <jlaidman at rebel-it.com.au>wrote:
On 14 November 2013 03:52, Asif Iqbal <vadud3 at gmail.com> wrote:
You might find that adjusting the TEST2RRD variable to include only
"columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
Looks like if I do not have columnname=ncv it does not generate a rrd file.
:-(
I did some clean up so nothing in MSG has NCV like data and I added the NCV data in the <-- ... --> . It did create the rrd file with the correct NCV keys, but values are zeros. It is dropping the values for some reason? So I am almost there if I can figure out why the values are not showing up.
Are the values you're reporting counter-type (that is, always incrementing) or gauge-type? If you're sending gauge data and it's being treated as counter data, then you will often get mostly zeroes or NaN. By default, NCV data is expected to be DERIVE which is a type of counter. If you want to change to GAUGE, then you need to set NCV_columnname to describe this:
NCV_columnname="foo:GAUGE,bar:GAUGE"
The other option would be using --extra-test and --extra-script and
parse the values from the status channel to generate rrd?
Perhaps. These seem a lot trickier to use.
One thought I had about this was to setup "--extra-test" and "--extra-script" on the status channel, to filter out the status messages from xymond_rrd. If you use "--extra-script=/bin/true" and "--extra-test=columnname" then I think it will feed the data to "/bin/true" instead of parsing it itself. Of course "/bin/true" will produce no output, and hence no data.
I want the graph show up in the test page, so sending it to just trends won't work, right?
Right. It needs to be listed in TEST2RRD to show up in the test page.
Hmm, just noticed this in the doco:
"No graph on the status page, but OK on the trends page Make sure you have *ncv* listed in the GRAPHS setting in xymonserver.cfg. (Don't ask why - just take my word that it must be there). "
Do you have "ncv" listed in GRAPHS?
J
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
On Wed, Nov 13, 2013 at 7:39 PM, Jeremy Laidman <jlaidman at rebel-it.com.au>wrote:
On 14 November 2013 03:52, Asif Iqbal <vadud3 at gmail.com> wrote:
You might find that adjusting the TEST2RRD variable to include only
"columname" and not "columnname=ncv" might do the trick. The man page for xymond_rrd talks about the "ncv" part only being used for status messages. But this is a long-shot.
Looks like if I do not have columnname=ncv it does not generate a rrd file.
:-(
I did some clean up so nothing in MSG has NCV like data and I added the NCV data in the <-- ... --> . It did create the rrd file with the correct NCV keys, but values are zeros. It is dropping the values for some reason? So I am almost there if I can figure out why the values are not showing up.
Are the values you're reporting counter-type (that is, always incrementing) or gauge-type? If you're sending gauge data and it's being treated as counter data, then you will often get mostly zeroes or NaN. By default, NCV data is expected to be DERIVE which is a type of counter. If you want to change to GAUGE, then you need to set NCV_columnname to describe this:
NCV_columnname="foo:GAUGE,bar:GAUGE"
Yes mine are GAUGE type and I have the NCV_columnname="*:GAUGE" setup
The other option would be using --extra-test and --extra-script and parse
the values from the status channel to generate rrd?
Perhaps. These seem a lot trickier to use.
One thought I had about this was to setup "--extra-test" and "--extra-script" on the status channel, to filter out the status messages from xymond_rrd. If you use "--extra-script=/bin/true" and "--extra-test=columnname" then I think it will feed the data to "/bin/true" instead of parsing it itself. Of course "/bin/true" will produce no output, and hence no data.
I can give that a try.
I want the graph show up in the test page, so sending it to just trends won't work, right?
Right. It needs to be listed in TEST2RRD to show up in the test page.
Hmm, just noticed this in the doco:
"No graph on the status page, but OK on the trends page Make sure you have *ncv* listed in the GRAPHS setting in xymonserver.cfg. (Don't ask why - just take my word that it must be there)."
Do you have "ncv" listed in GRAPHS?
Yes I have the columname in GRAPHS and columnname=ncv in TEST2RRD
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
participants (3)
-
everett.vernon＠gmail.com
-
jlaidman＠rebel-it.com.au
-
vadud3＠gmail.com