I've got a new test that I need to graph. It is a simple count of devices in multiple "networks". This test only runs once a day. I haven't done a graph in xymon in several years, and it's always a struggle.
This is the input data:
Net_AtoD: 5871 Net_EtoN: 5589 Net_OtoZ_1to9: 5242 Other: 386 InfraGen: 12 InfraSec: 25
The data comes in fine. And I get a good rrd file (all 6 networks are in there and look the same as this, except obviously the last_ds and value values: <ds> <name> NetAtoD </name> <type> GAUGE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max>
<!-- PDP Status -->
<last_ds> 5871 </last_ds>
<value> 5.8710000000e+04 </value>
<unknown_sec> 267 </unknown_sec>
</ds>I have multiple xymon servers, and I share most of the config with an include line in xymonserver.cfg:
/etc/xymon/xymonserver-common.cfg:TEST2RRD+=",proc-mem-cfgmgrd=ncv,ucf=ncv,ConfigPulls=ncv" /etc/xymon/xymonserver-common.cfg:GRAPHS+=",vmwarevms,hpnasnaphost,ucf,ConfigPulls" /etc/xymon/xymonserver-common.cfg:NCV_ConfigPulls="*:GAUGE"
I've restarted a few times.
I put the graph config in its own file /etc/xymon/graph.d/ConfigPulls.cfg [ConfigPulls] TITLE NCM Config PUlls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:EtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:OtoZ:AVERAGE DEF:Other=ConfigPUlls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPUlls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPUlls.rrd:InfraSec:AVERAGE LINE2:AtoD#00FF00:Net A to D LINE2:EtoN#00CCCC:Net E to N LINE2:OtoZ#FFCC00:Net O to Z & 1 to 9 LINE2:Government#FF0000:Government LINE2:InfraGen#FFFF00:Infrastructure General LINE2:InfraSec#FFFFCC:Infrastructure Secure COMMENT:\n GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur) GPRINT:AtoD:MAX: \: %5.1lf (max) GPRINT:AtoD:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:EtoN:LAST:Net E to N \: %5.1lf (cur) GPRINT:EtoN:MAX: \: %5.1lf (max) GPRINT:EtoN:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:OtoZ:LAST:Net 0 to Z \: %5.1lf (cur) GPRINT:OtoZ:MAX: \: %5.1lf (max) GPRINT:OtoZ:MIN: \: %5.1lf (min) GPRINT:OtoZ:AVERAGE: \: %5.1lf (avg)\n GPRINT:Government:LAST:Government \: %5.1lf (cur) GPRINT:Government:MAX: \: %5.1lf (max) GPRINT:Government:MIN: \: %5.1lf (min) GPRINT:Government:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraGen:LAST:Infrastructure General \: %5.1lf (cur) GPRINT:InfraGen:MAX: \: %5.1lf (max) GPRINT:InfraGen:MIN: \: %5.1lf (min) GPRINT:InfraGen:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraSec:LAST:Infrastructure Secure \: %5.1lf (cur) GPRINT:InfraSec:MAX: \: %5.1lf (max) GPRINT:InfraSec:MIN: \: %5.1lf (min) GPRINT:InfraSec:AVERAGE: \: %5.1lf (avg)\n
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help?
Thanks, Paul.
Paul Root Lead Operations Engineer - IT Managed Services 390 Commerce Dr Woodbury, Mn 55125 651-312-5207 paul.root at centurylink.com
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
I think maybe your names don't match??
<ds>
<name> NetAtoD </name>not equal to:
DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE
.....
LINE2:AtoD#00FF00:Net A to D
.....
GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur)I think it should be:
DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE
....
LINE2:NetAtoD#00FF00:Net A to D
....
GPRINT:NetAtoD:LAST:Net A to D \: %5.1lf (cur)and similarly for the other DEF, LINE2 and GPRINT. It's been a long time since I messed with graphs too, so I could be completely wrong.
Ralph Mitchell
On Wed, Sep 4, 2019 at 1:59 PM Root, Paul T <Paul.Root at centurylink.com> wrote:
I?ve got a new test that I need to graph. It is a simple count of devices in multiple ?networks?. This test only runs once a day. I haven?t done a graph in xymon in several years, and it?s always a struggle.
This is the input data:
Net_AtoD: 5871 Net_EtoN: 5589 Net_OtoZ_1to9: 5242 Other: 386 InfraGen: 12 InfraSec: 25
The data comes in fine. And I get a good rrd file (all 6 networks are in there and look the same as this, except obviously the last_ds and value values:
<ds> <name> NetAtoD </name> <type> GAUGE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max> <!-- PDP Status --> <last_ds> 5871 </last_ds> <value> 5.8710000000e+04 </value> <unknown_sec> 267 </unknown_sec> </ds>I have multiple xymon servers, and I share most of the config with an include line in xymonserver.cfg:
/etc/xymon/xymonserver-common.cfg:TEST2RRD+=",proc-mem-cfgmgrd=ncv,ucf=ncv,ConfigPulls=ncv"
/etc/xymon/xymonserver-common.cfg:GRAPHS+=",vmwarevms,hpnasnaphost,ucf,ConfigPulls"
/etc/xymon/xymonserver-common.cfg:NCV_ConfigPulls="*:GAUGE"
I?ve restarted a few times.
I put the graph config in its own file /etc/xymon/graph.d/ConfigPulls.cfg
[ConfigPulls]
TITLE NCM Config PUlls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:EtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:OtoZ:AVERAGE DEF:Other=ConfigPUlls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPUlls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPUlls.rrd:InfraSec:AVERAGE LINE2:AtoD#00FF00:Net A to D LINE2:EtoN#00CCCC:Net E to N LINE2:OtoZ#FFCC00:Net O to Z & 1 to 9 LINE2:Government#FF0000:Government LINE2:InfraGen#FFFF00:Infrastructure General LINE2:InfraSec#FFFFCC:Infrastructure Secure COMMENT:\n GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur) GPRINT:AtoD:MAX: \: %5.1lf (max) GPRINT:AtoD:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:EtoN:LAST:Net E to N \: %5.1lf (cur) GPRINT:EtoN:MAX: \: %5.1lf (max) GPRINT:EtoN:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:OtoZ:LAST:Net 0 to Z \: %5.1lf (cur) GPRINT:OtoZ:MAX: \: %5.1lf (max) GPRINT:OtoZ:MIN: \: %5.1lf (min) GPRINT:OtoZ:AVERAGE: \: %5.1lf (avg)\n GPRINT:Government:LAST:Government \: %5.1lf (cur) GPRINT:Government:MAX: \: %5.1lf (max) GPRINT:Government:MIN: \: %5.1lf (min) GPRINT:Government:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraGen:LAST:Infrastructure General \: %5.1lf (cur) GPRINT:InfraGen:MAX: \: %5.1lf (max) GPRINT:InfraGen:MIN: \: %5.1lf (min) GPRINT:InfraGen:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraSec:LAST:Infrastructure Secure \: %5.1lf (cur) GPRINT:InfraSec:MAX: \: %5.1lf (max) GPRINT:InfraSec:MIN: \: %5.1lf (min) GPRINT:InfraSec:AVERAGE: \: %5.1lf (avg)\nThe graph doesn?t show up, just a broken link in both the test and trends pages.
Any help?
Thanks,
Paul.
Paul Root
Lead Operations Engineer - IT Managed Services
390 Commerce Dr
Woodbury, Mn 55125
651-312-5207 paul.root at centurylink.com
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Ah, thanks. I?ll give that a try. I was going quick and thinking I was relating a file to a field, and then putting a label in.
From: Ralph Mitchell <ralphmitchell at gmail.com> Sent: Wednesday, September 04, 2019 1:22 PM To: Root, Paul T <Paul.Root at CenturyLink.com> Cc: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
I think maybe your names don't match??
<ds>
<name> NetAtoD </name>not equal to:
DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE
.....
LINE2:AtoD#00FF00:Net A to D
.....
GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur)I think it should be:
DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE
....
LINE2:NetAtoD#00FF00:Net A to D
....
GPRINT:NetAtoD:LAST:Net A to D \: %5.1lf (cur)and similarly for the other DEF, LINE2 and GPRINT. It's been a long time since I messed with graphs too, so I could be completely wrong.
Ralph Mitchell
On Wed, Sep 4, 2019 at 1:59 PM Root, Paul T <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: I?ve got a new test that I need to graph. It is a simple count of devices in multiple ?networks?. This test only runs once a day. I haven?t done a graph in xymon in several years, and it?s always a struggle.
This is the input data:
Net_AtoD: 5871 Net_EtoN: 5589 Net_OtoZ_1to9: 5242 Other: 386 InfraGen: 12 InfraSec: 25
The data comes in fine. And I get a good rrd file (all 6 networks are in there and look the same as this, except obviously the last_ds and value values: <ds> <name> NetAtoD </name> <type> GAUGE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max>
<!-- PDP Status -->
<last_ds> 5871 </last_ds>
<value> 5.8710000000e+04 </value>
<unknown_sec> 267 </unknown_sec>
</ds>I have multiple xymon servers, and I share most of the config with an include line in xymonserver.cfg:
/etc/xymon/xymonserver-common.cfg:TEST2RRD+=",proc-mem-cfgmgrd=ncv,ucf=ncv,ConfigPulls=ncv" /etc/xymon/xymonserver-common.cfg:GRAPHS+=",vmwarevms,hpnasnaphost,ucf,ConfigPulls" /etc/xymon/xymonserver-common.cfg:NCV_ConfigPulls="*:GAUGE"
I?ve restarted a few times.
I put the graph config in its own file /etc/xymon/graph.d/ConfigPulls.cfg [ConfigPulls] TITLE NCM Config PUlls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:EtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:OtoZ:AVERAGE DEF:Other=ConfigPUlls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPUlls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPUlls.rrd:InfraSec:AVERAGE LINE2:AtoD#00FF00:Net A to D LINE2:EtoN#00CCCC:Net E to N LINE2:OtoZ#FFCC00:Net O to Z & 1 to 9 LINE2:Government#FF0000:Government LINE2:InfraGen#FFFF00:Infrastructure General LINE2:InfraSec#FFFFCC:Infrastructure Secure COMMENT:\n GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur) GPRINT:AtoD:MAX: \: %5.1lf (max) GPRINT:AtoD:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:EtoN:LAST:Net E to N \: %5.1lf (cur) GPRINT:EtoN:MAX: \: %5.1lf (max) GPRINT:EtoN:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:OtoZ:LAST:Net 0 to Z \: %5.1lf (cur) GPRINT:OtoZ:MAX: \: %5.1lf (max) GPRINT:OtoZ:MIN: \: %5.1lf (min) GPRINT:OtoZ:AVERAGE: \: %5.1lf (avg)\n GPRINT:Government:LAST:Government \: %5.1lf (cur) GPRINT:Government:MAX: \: %5.1lf (max) GPRINT:Government:MIN: \: %5.1lf (min) GPRINT:Government:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraGen:LAST:Infrastructure General \: %5.1lf (cur) GPRINT:InfraGen:MAX: \: %5.1lf (max) GPRINT:InfraGen:MIN: \: %5.1lf (min) GPRINT:InfraGen:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraSec:LAST:Infrastructure Secure \: %5.1lf (cur) GPRINT:InfraSec:MAX: \: %5.1lf (max) GPRINT:InfraSec:MIN: \: %5.1lf (min) GPRINT:InfraSec:AVERAGE: \: %5.1lf (avg)\n
The graph doesn?t show up, just a broken link in both the test and trends pages.
Any help?
Thanks, Paul.
Paul Root Lead Operations Engineer - IT Managed Services 390 Commerce Dr Woodbury, Mn 55125 651-312-5207 paul.root at centurylink.com<mailto:paul.root at centurylink.com>
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Well, this didn?t work.
I?ve removed the file from the /etc/xymon/graph.d directory, and added it directly into graphs.cfg. No help.
I removed the comment and GPRINT lines, as they are superfluous at this time. Still nothing.
Is there a way to manually create a gif/jpg/whatever with rrdtool to test that the data will create a graph?
What I have now: [ConfigPulls] TITLE NCM Config Pulls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:NetOtoZ:AVERAGE DEF:Other=ConfigPulls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE
As I read the how to, the DEF is DEF:<label you want>=<rrdfile>:<data label in the rrd file:AVERAGE
So those should be right.
From: Ralph Mitchell <ralphmitchell at gmail.com> Sent: Wednesday, September 04, 2019 1:22 PM To: Root, Paul T <Paul.Root at CenturyLink.com> Cc: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
I think maybe your names don't match??
<ds>
<name> NetAtoD </name>not equal to:
DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE
.....
LINE2:AtoD#00FF00:Net A to D
.....
GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur)I think it should be:
DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE
....
LINE2:NetAtoD#00FF00:Net A to D
....
GPRINT:NetAtoD:LAST:Net A to D \: %5.1lf (cur)and similarly for the other DEF, LINE2 and GPRINT. It's been a long time since I messed with graphs too, so I could be completely wrong.
Ralph Mitchell
On Wed, Sep 4, 2019 at 1:59 PM Root, Paul T <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: I?ve got a new test that I need to graph. It is a simple count of devices in multiple ?networks?. This test only runs once a day. I haven?t done a graph in xymon in several years, and it?s always a struggle.
This is the input data:
Net_AtoD: 5871 Net_EtoN: 5589 Net_OtoZ_1to9: 5242 Other: 386 InfraGen: 12 InfraSec: 25
The data comes in fine. And I get a good rrd file (all 6 networks are in there and look the same as this, except obviously the last_ds and value values: <ds> <name> NetAtoD </name> <type> GAUGE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max>
<!-- PDP Status -->
<last_ds> 5871 </last_ds>
<value> 5.8710000000e+04 </value>
<unknown_sec> 267 </unknown_sec>
</ds>I have multiple xymon servers, and I share most of the config with an include line in xymonserver.cfg:
/etc/xymon/xymonserver-common.cfg:TEST2RRD+=",proc-mem-cfgmgrd=ncv,ucf=ncv,ConfigPulls=ncv" /etc/xymon/xymonserver-common.cfg:GRAPHS+=",vmwarevms,hpnasnaphost,ucf,ConfigPulls" /etc/xymon/xymonserver-common.cfg:NCV_ConfigPulls="*:GAUGE"
I?ve restarted a few times.
I put the graph config in its own file /etc/xymon/graph.d/ConfigPulls.cfg [ConfigPulls] TITLE NCM Config PUlls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:EtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:OtoZ:AVERAGE DEF:Other=ConfigPUlls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPUlls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPUlls.rrd:InfraSec:AVERAGE LINE2:AtoD#00FF00:Net A to D LINE2:EtoN#00CCCC:Net E to N LINE2:OtoZ#FFCC00:Net O to Z & 1 to 9 LINE2:Government#FF0000:Government LINE2:InfraGen#FFFF00:Infrastructure General LINE2:InfraSec#FFFFCC:Infrastructure Secure COMMENT:\n GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur) GPRINT:AtoD:MAX: \: %5.1lf (max) GPRINT:AtoD:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:EtoN:LAST:Net E to N \: %5.1lf (cur) GPRINT:EtoN:MAX: \: %5.1lf (max) GPRINT:EtoN:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:OtoZ:LAST:Net 0 to Z \: %5.1lf (cur) GPRINT:OtoZ:MAX: \: %5.1lf (max) GPRINT:OtoZ:MIN: \: %5.1lf (min) GPRINT:OtoZ:AVERAGE: \: %5.1lf (avg)\n GPRINT:Government:LAST:Government \: %5.1lf (cur) GPRINT:Government:MAX: \: %5.1lf (max) GPRINT:Government:MIN: \: %5.1lf (min) GPRINT:Government:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraGen:LAST:Infrastructure General \: %5.1lf (cur) GPRINT:InfraGen:MAX: \: %5.1lf (max) GPRINT:InfraGen:MIN: \: %5.1lf (min) GPRINT:InfraGen:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraSec:LAST:Infrastructure Secure \: %5.1lf (cur) GPRINT:InfraSec:MAX: \: %5.1lf (max) GPRINT:InfraSec:MIN: \: %5.1lf (min) GPRINT:InfraSec:AVERAGE: \: %5.1lf (avg)\n
The graph doesn?t show up, just a broken link in both the test and trends pages.
Any help?
Thanks, Paul.
Paul Root Lead Operations Engineer - IT Managed Services 390 Commerce Dr Woodbury, Mn 55125 651-312-5207 paul.root at centurylink.com<mailto:paul.root at centurylink.com>
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
I got a two-line graph from the netstat.rrd by doing this:
rrdtool graph netstat.jpg \
"DEF:udpInDatagrams=netstat.rrd:udpInDatagrams:AVERAGE" \
"LINE3:udpInDatagrams#00FF00:UDP In Datagrams" \
"DEF:udpOutDatagrams=netstat.rrd:udpOutDatagrams:AVERAGE" \
"LINE1:udpOutDatagrams#0000FF:UDP Out Datagrams"The quoted DEF and LINE strings are directly copied from the [netstat1] config in the graph.cfg file.
Ralph Mitchell
On Thu, Sep 5, 2019 at 12:32 PM Root, Paul T <Paul.Root at centurylink.com> wrote:
Well, this didn?t work.
I?ve removed the file from the /etc/xymon/graph.d directory, and added it directly into graphs.cfg. No help.
I removed the comment and GPRINT lines, as they are superfluous at this time. Still nothing.
Is there a way to manually create a gif/jpg/whatever with rrdtool to test that the data will create a graph?
What I have now:
[ConfigPulls]
TITLE NCM Config Pulls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:NetOtoZ:AVERAGE DEF:Other=ConfigPulls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGEAs I read the how to, the DEF is
DEF:<label you want>=<rrdfile>:<data label in the rrd file:AVERAGE
So those should be right.
*From:* Ralph Mitchell <ralphmitchell at gmail.com> *Sent:* Wednesday, September 04, 2019 1:22 PM *To:* Root, Paul T <Paul.Root at CenturyLink.com> *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] Graphing in Xymon
I think maybe your names don't match??
<ds> <name> NetAtoD </name>not equal to:
DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE ..... LINE2:AtoD#00FF00:Net A to D ..... GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur)I think it should be:
DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE .... LINE2:NetAtoD#00FF00:Net A to D .... GPRINT:NetAtoD:LAST:Net A to D \: %5.1lf (cur)and similarly for the other DEF, LINE2 and GPRINT. It's been a long time since I messed with graphs too, so I could be completely wrong.
Ralph Mitchell
On Wed, Sep 4, 2019 at 1:59 PM Root, Paul T <Paul.Root at centurylink.com> wrote:
I?ve got a new test that I need to graph. It is a simple count of devices in multiple ?networks?. This test only runs once a day. I haven?t done a graph in xymon in several years, and it?s always a struggle.
This is the input data:
Net_AtoD: 5871 Net_EtoN: 5589 Net_OtoZ_1to9: 5242 Other: 386 InfraGen: 12 InfraSec: 25
The data comes in fine. And I get a good rrd file (all 6 networks are in there and look the same as this, except obviously the last_ds and value values:
<ds> <name> NetAtoD </name> <type> GAUGE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max> <!-- PDP Status --> <last_ds> 5871 </last_ds> <value> 5.8710000000e+04 </value> <unknown_sec> 267 </unknown_sec> </ds>I have multiple xymon servers, and I share most of the config with an include line in xymonserver.cfg:
/etc/xymon/xymonserver-common.cfg:TEST2RRD+=",proc-mem-cfgmgrd=ncv,ucf=ncv,ConfigPulls=ncv"
/etc/xymon/xymonserver-common.cfg:GRAPHS+=",vmwarevms,hpnasnaphost,ucf,ConfigPulls"
/etc/xymon/xymonserver-common.cfg:NCV_ConfigPulls="*:GAUGE"
I?ve restarted a few times.
I put the graph config in its own file /etc/xymon/graph.d/ConfigPulls.cfg
[ConfigPulls]
TITLE NCM Config PUlls YAXIS Number of Devices DEF:NetAtoD=ConfigPulls.rrd:AtoD:AVERAGE DEF:NetEtoN=ConfigPulls.rrd:EtoN:AVERAGE DEF:NetOtoZ=ConfigPulls.rrd:OtoZ:AVERAGE DEF:Other=ConfigPUlls.rrd:Other:AVERAGE DEF:InfraGen=ConfigPUlls.rrd:InfraGen:AVERAGE DEF:InfraSec=ConfigPUlls.rrd:InfraSec:AVERAGE LINE2:AtoD#00FF00:Net A to D LINE2:EtoN#00CCCC:Net E to N LINE2:OtoZ#FFCC00:Net O to Z & 1 to 9 LINE2:Government#FF0000:Government LINE2:InfraGen#FFFF00:Infrastructure General LINE2:InfraSec#FFFFCC:Infrastructure Secure COMMENT:\n GPRINT:AtoD:LAST:Net A to D \: %5.1lf (cur) GPRINT:AtoD:MAX: \: %5.1lf (max) GPRINT:AtoD:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:EtoN:LAST:Net E to N \: %5.1lf (cur) GPRINT:EtoN:MAX: \: %5.1lf (max) GPRINT:EtoN:MIN: \: %5.1lf (min) GPRINT:EtoN:AVERAGE: \: %5.1lf (avg)\n GPRINT:OtoZ:LAST:Net 0 to Z \: %5.1lf (cur) GPRINT:OtoZ:MAX: \: %5.1lf (max) GPRINT:OtoZ:MIN: \: %5.1lf (min) GPRINT:OtoZ:AVERAGE: \: %5.1lf (avg)\n GPRINT:Government:LAST:Government \: %5.1lf (cur) GPRINT:Government:MAX: \: %5.1lf (max) GPRINT:Government:MIN: \: %5.1lf (min) GPRINT:Government:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraGen:LAST:Infrastructure General \: %5.1lf (cur) GPRINT:InfraGen:MAX: \: %5.1lf (max) GPRINT:InfraGen:MIN: \: %5.1lf (min) GPRINT:InfraGen:AVERAGE: \: %5.1lf (avg)\n GPRINT:InfraSec:LAST:Infrastructure Secure \: %5.1lf (cur) GPRINT:InfraSec:MAX: \: %5.1lf (max) GPRINT:InfraSec:MIN: \: %5.1lf (min) GPRINT:InfraSec:AVERAGE: \: %5.1lf (avg)\nThe graph doesn?t show up, just a broken link in both the test and trends pages.
Any help?
Thanks,
Paul.
Paul Root
Lead Operations Engineer - IT Managed Services
390 Commerce Dr
Woodbury, Mn 55125
651-312-5207 paul.root at centurylink.com
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn?t show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"
-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph
Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Oh, the client machine is RedHat 6 and xymon-client 4.3.28
-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Root, Paul T Sent: Thursday, September 05, 2019 3:17 PM To: 'Japheth Cleaver' <cleaver at terabithia.org>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph
Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Looking in the rrd file, it doesn't seem like there are the historical data in the file.
The data is only going to change once a day, so I was only calling the script once a day.
I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data.
Does the graphing thing need more frequent data?
-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Could be wrong, but I think the graphs need two or three data points in order to show anything.
Ralph Mitchell
On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <Paul.Root at centurylink.com> wrote:
Looking in the rrd file, it doesn't seem like there are the historical data in the file.
The data is only going to change once a day, so I was only calling the script once a day.
I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data.
Does the graphing thing need more frequent data?
-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It onlyhas 1 data point at this time, but again. Running rrdtool graph on that file produces a graph
Oh, I never mentioned before. This is CentOS 6 with my ownbuild for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.
Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget " http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... " --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
I don?t think the rrd file is working right.
It is there, and the latest numbers are in, but I don?t see any archive numbers populating.
Where can I check for that?
From: Ralph Mitchell <ralphmitchell at gmail.com> Sent: Friday, September 06, 2019 10:07 AM To: Root, Paul T <Paul.Root at CenturyLink.com> Cc: Japheth Cleaver <cleaver at terabithia.org>; Stef Coene <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
Could be wrong, but I think the graphs need two or three data points in order to show anything.
Ralph Mitchell
On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: Looking in the rrd file, it doesn't seem like there are the historical data in the file.
The data is only going to change once a day, so I was only calling the script once a day.
I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data.
Does the graphing thing need more frequent data?
-----Original Message----- From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org<mailto:cleaver at terabithia.org>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
You should see four different RRA's (round-robin archives) when you run 'rrdtool info' on the actual RRD file, each being populated as "roll-up" averages of the various data point. For example: ]# rrdtool info clock.rrd filename = "clock.rrd" rrd_version = "0003" step = 300 last_update = 1568049660 header_size = 1208 ds[la].index = 0 ds[la].type = "GAUGE" ds[la].minimal_heartbeat = 600 ds[la].min = NaN ds[la].max = NaN ds[la].last_ds = "0" ds[la].value = 0.0000000000e+00 ds[la].unknown_sec = 0 rra[0].cf = "AVERAGE" rra[0].rows = 576 rra[0].cur_row = 253 rra[0].pdp_per_row = 1 rra[0].xff = 5.0000000000e-01 rra[0].cdp_prep[0].value = NaN rra[0].cdp_prep[0].unknown_datapoints = 0 rra[1].cf = "AVERAGE" rra[1].rows = 576 rra[1].cur_row = 419 rra[1].pdp_per_row = 6 rra[1].xff = 5.0000000000e-01 rra[1].cdp_prep[0].value = 0.0000000000e+00 rra[1].cdp_prep[0].unknown_datapoints = 0 rra[2].cf = "AVERAGE" rra[2].rows = 576 rra[2].cur_row = 520 rra[2].pdp_per_row = 24 rra[2].xff = 5.0000000000e-01 rra[2].cdp_prep[0].value = 0.0000000000e+00 rra[2].cdp_prep[0].unknown_datapoints = 0 rra[3].cf = "AVERAGE" rra[3].rows = 576 rra[3].cur_row = 392 rra[3].pdp_per_row = 288 rra[3].xff = 5.0000000000e-01 rra[3].cdp_prep[0].value = 3.4666666667e-01 rra[3].cdp_prep[0].unknown_datapoints = 0
I suspect the issue here is the lack of data in the originating source. It seems like a once-a-day datapoint should at least be made visible in the 2d RRA, but I've not actually tried to configure one with low frequency before. Would you be able to artificially submit that point several times within a 5m period and see if it populates? Keeping --no-cache back on may be beneficial here too.
Regards, -jc
On 9/6/2019 11:01 AM, Root, Paul T wrote:
I don?t think the rrd file is working right.
It is there, and the latest numbers are in, but I don?t see any archive numbers populating.
Where can I check for that?
*From:*Ralph Mitchell <ralphmitchell at gmail.com> *Sent:* Friday, September 06, 2019 10:07 AM *To:* Root, Paul T <Paul.Root at CenturyLink.com> *Cc:* Japheth Cleaver <cleaver at terabithia.org>; Stef Coene <stef.coene at docum.org>; xymon at xymon.com *Subject:* Re: [Xymon] Graphing in Xymon
Could be wrong, but I think the graphs need two or three data points in order to show anything.
Ralph Mitchell
On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <Paul.Root at centurylink.com <mailto:Paul.Root at centurylink.com>> wrote:
Looking in the rrd file, it doesn't seem like there are the historical data in the file. The data is only going to change once a day, so I was only calling the script once a day. I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data. Does the graphing thing need more frequent data? -----Original Message----- From: Xymon <xymon-bounces at xymon.com <mailto:xymon-bounces at xymon.com>> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org <mailto:cleaver at terabithia.org>>; 'Stef Coene' <stef.coene at docum.org <mailto:stef.coene at docum.org>>; xymon at xymon.com <mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon No difference. I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes. -----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org <mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com <mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org <mailto:stef.coene at docum.org>>; xymon at xymon.com <mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon Hmm. That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance? -jc On 9/5/2019 1:48 PM, Root, Paul T wrote: > Those files look ok > > $ fuser * > rrdctl.2339:? ? ? ? ? 2339 > rrdctl.2437:? ? ? ? ? 2437 > $ ps -ef |grep 2339 > xymon? ? ?2339? 2323? 0 15:09 ?? ? ? ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache > root? ? ? 5083? 4674? 0 15:43 pts/0? ? 00:00:00 grep 2339 > $ ps -ef |grep 2437 > xymon? ? ?2437? 2324? 0 15:09 ?? ? ? ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache > root? ? ? 5086? 4674? 0 15:43 pts/0? ? 00:00:00 grep 2437 > $ ls -la > total 19964 > drwxr-xr-x.? 2 xymon xymon? ? ?4096 Sep? 5 15:39 . > drwxr-xr-x. 12 xymon xymon? ? ?4096 Mar? 1? 2016 .. > -rw-rw-r--.? 1 xymon xymon? ? 52136 Sep? 5 15:43 alert.chk > -rw-rw-r--.? 1 xymon xymon? ? ? ?96 Sep? 5 15:43 alert.chk.sub > -rw-r--r--.? 1 xymon xymon? ? ? ?24 Sep? 5 15:39 ping..status > srw-rw-rw-.? 1 xymon xymon? ? ? ? 0 Sep? 5 15:09 rrdctl.2339 > srw-rw-rw-.? 1 xymon xymon? ? ? ? 0 Sep? 5 15:09 rrdctl.2437 > -rw-r--r--.? 1 xymon xymon? ? ? ?22 Oct 19? 2018 ssh_8022..status > -rw-r--r--.? 1 xymon xymon 20366660 Sep? 5 15:39 xymond.chk > > Selinux is set to permissive. > > Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well. > > -----Original Message----- > From: Japheth Cleaver <cleaver at terabithia.org <mailto:cleaver at terabithia.org>> > Sent: Thursday, September 05, 2019 3:33 PM > To: Root, Paul T <Paul.Root at CenturyLink.com <mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org <mailto:stef.coene at docum.org>>; xymon at xymon.com <mailto:xymon at xymon.com> > Subject: Re: [Xymon] Graphing in Xymon > > This is probably a sign of invalid rrdctl socket files, as that's what > showgraph is trying to use to communicate with xymond_rrd. > > These should be in the TMP/RUN dir on a source compile, named > "rrdctl.<pid>", one for each of the currently running xymond_rrd > processes (usually two,m one for status, one for data). They'll need to > be writable by your apache user at least, and SELinux may require > additional permissions for the sendto if you have it enabled and > enforcing. If you see lots of stale ones, be sure to remove them -- as > showgraph.sh doesn't know what's current, I believe it tries to cycle > through all of them, which could end up in timeouts. > > Regardless, premature header ending isn't a desired result no matter > what the situation. Trying to find all cases of this is a longer term > goal for all the CGIs. > > HTH, > -jc > > On 9/5/2019 1:16 PM, Root, Paul T wrote: >> Ah, >>? ? ? ? ? ?I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors: >> >> >> [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls >> [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls >> [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls >> >>? ? ? ? ? ?And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph >> >> >>? ? ? ? ? ?Oh, I never mentioned before. ?This is CentOS 6 with my own build for 4.3.21.? I'm starting to move to the Terabithia repo, but haven't got there yet on the server. >> >> Paul. >> >> >> >> -----Original Message----- >> From: Japheth Cleaver <cleaver at terabithia.org <mailto:cleaver at terabithia.org>> >> Sent: Thursday, September 05, 2019 3:00 PM >> To: Root, Paul T <Paul.Root at CenturyLink.com <mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org <mailto:stef.coene at docum.org>>; xymon at xymon.com <mailto:xymon at xymon.com> >> Subject: Re: [Xymon] Graphing in Xymon >> >> A 500 ISE isn't a good sign. Is there anything in the web error logs? >> >> And just to confirm, the file does actually existing in the RRD >> directory as expected? >> >> Regards, >> -jc >> >> On 9/5/2019 12:56 PM, Root, Paul T wrote: >>> Not much help here: >>> >>> $ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view" >>> --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view >>> Resolving nsmdenvp117.corp.intranet... 151.119.9.117 >>> Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. >>> HTTP request sent, awaiting response... 500 Internal Server Error >>> 2019-09-05 14:46:59 ERROR 500: Internal Server Error. >>> >>> >>> Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph. >>> >>> rrdtool graph ConfigPulls.jpg \ >>>? ? ? ? ? ? --title "NCM Config Pulls" \ >>> "DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE" \ >>> "DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE" \ >>> "DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE" \ >>> "DEF:Other=ConfigPulls.rrd:Other:AVERAGE" \ >>> "DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE" \ >>> "DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE" \ >>>? ? ? ? ? ? "LINE2:NetAtoD#00FF00:Net A to D" \ >>>? ? ? ? ? ? "LINE2:NetEtoN#00CCCC:Net E to N" \ >>>? ? ? ? ? ? "LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9" \ >>>? ? ? ? ? ? "LINE2:Other#FF0000:Other" \ >>> "LINE2:InfraGen#FFFF00:Infrastructure General" \ >>> "LINE2:InfraSec#FFFFCC:Infrastructure Secure" \ >>>? ? ? ? ? ? "COMMENT:\n" >>> >>> >>> >>> -----Original Message----- >>> From: Xymon <xymon-bounces at xymon.com <mailto:xymon-bounces at xymon.com>> On Behalf Of Stef Coene >>> Sent: Thursday, September 05, 2019 2:18 PM >>> To: xymon at xymon.com <mailto:xymon at xymon.com> >>> Subject: Re: [Xymon] Graphing in Xymon >>> >>> On 4/09/19 19:59, Root, Paul T wrote: >>>> The graph doesn't show up, just a broken link in both the test and >>>> trends pages. >>>> >>>> Any help? >>> Then there is a problem generating the RRD, the error is returned as >>> text. But you browser expects a graph, not some text so it displays a >>> broken image symbol. >>> >>> When yoy right click on one of the broken image, you can say 'open image >>> in new tab'. >>> This will give you the direct URL of the graph. >>> If you do a wget of this URL (don't forget quotes!) on your xymon >>> server, you will download the real error. >>> Just do a vi of the downloaded file to see this error. >>> >>> >>> Stef This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments. _______________________________________________ Xymon mailing list Xymon at xymon.com <mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments. _______________________________________________ Xymon mailing list Xymon at xymon.com <mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymonThis communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
I do see 4 archives: filename = "ConfigPulls.rrd" rrd_version = "0003" step = 300 last_update = 1568044491 ds[NetAtoD].type = "GAUGE" ds[NetAtoD].minimal_heartbeat = 600 ds[NetAtoD].min = NaN ds[NetAtoD].max = NaN ds[NetAtoD].last_ds = "U" ds[NetAtoD].value = 0.0000000000e+00 ds[NetAtoD].unknown_sec = 291 ds[NetOtoZ1to9].type = "GAUGE" ds[NetOtoZ1to9].minimal_heartbeat = 600 ds[NetOtoZ1to9].min = NaN ds[NetOtoZ1to9].max = NaN ds[NetOtoZ1to9].last_ds = "U" ds[NetOtoZ1to9].value = 0.0000000000e+00 ds[NetOtoZ1to9].unknown_sec = 291 ds[Other].type = "GAUGE" ds[Other].minimal_heartbeat = 600 ds[Other].min = NaN ds[Other].max = NaN ds[Other].last_ds = "U" ds[Other].value = 0.0000000000e+00 ds[Other].unknown_sec = 291 ds[InfraGen].type = "GAUGE" ds[InfraGen].minimal_heartbeat = 600 ds[InfraGen].min = NaN ds[InfraGen].max = NaN ds[InfraGen].last_ds = "U" ds[InfraGen].value = 0.0000000000e+00 ds[InfraGen].unknown_sec = 291 ds[InfraSec].type = "GAUGE" ds[InfraSec].minimal_heartbeat = 600 ds[InfraSec].min = NaN ds[InfraSec].max = NaN ds[InfraSec].last_ds = "U" ds[InfraSec].value = 0.0000000000e+00 ds[InfraSec].unknown_sec = 291 rra[0].cf = "AVERAGE" rra[0].rows = 576 rra[0].cur_row = 234 rra[0].pdp_per_row = 1 rra[0].xff = 5.0000000000e-01 rra[0].cdp_prep[0].value = NaN rra[0].cdp_prep[0].unknown_datapoints = 0 rra[0].cdp_prep[1].value = NaN rra[0].cdp_prep[1].unknown_datapoints = 0 rra[0].cdp_prep[2].value = NaN rra[0].cdp_prep[2].unknown_datapoints = 0 rra[0].cdp_prep[3].value = NaN rra[0].cdp_prep[3].unknown_datapoints = 0 rra[0].cdp_prep[4].value = NaN rra[0].cdp_prep[4].unknown_datapoints = 0 rra[1].cf = "AVERAGE" rra[1].rows = 576 rra[1].cur_row = 337 rra[1].pdp_per_row = 6 rra[1].xff = 5.0000000000e-01 rra[1].cdp_prep[0].value = NaN rra[1].cdp_prep[0].unknown_datapoints = 4 rra[1].cdp_prep[1].value = NaN rra[1].cdp_prep[1].unknown_datapoints = 4 rra[1].cdp_prep[2].value = NaN rra[1].cdp_prep[2].unknown_datapoints = 4 rra[1].cdp_prep[3].value = NaN rra[1].cdp_prep[3].unknown_datapoints = 4 rra[1].cdp_prep[4].value = NaN rra[1].cdp_prep[4].unknown_datapoints = 4 rra[2].cf = "AVERAGE" rra[2].rows = 576 rra[2].cur_row = 227 rra[2].pdp_per_row = 24 rra[2].xff = 5.0000000000e-01 rra[2].cdp_prep[0].value = NaN rra[2].cdp_prep[0].unknown_datapoints = 22 rra[2].cdp_prep[1].value = NaN rra[2].cdp_prep[1].unknown_datapoints = 22 rra[2].cdp_prep[2].value = NaN rra[2].cdp_prep[2].unknown_datapoints = 22 rra[2].cdp_prep[3].value = NaN rra[2].cdp_prep[3].unknown_datapoints = 22 rra[2].cdp_prep[4].value = NaN rra[2].cdp_prep[4].unknown_datapoints = 22 rra[3].cf = "AVERAGE" rra[3].rows = 576 rra[3].cur_row = 172 rra[3].pdp_per_row = 288 rra[3].xff = 5.0000000000e-01 rra[3].cdp_prep[0].value = NaN rra[3].cdp_prep[0].unknown_datapoints = 190 rra[3].cdp_prep[1].value = NaN rra[3].cdp_prep[1].unknown_datapoints = 190 rra[3].cdp_prep[2].value = NaN rra[3].cdp_prep[2].unknown_datapoints = 190 rra[3].cdp_prep[3].value = NaN rra[3].cdp_prep[3].unknown_datapoints = 190 rra[3].cdp_prep[4].value = NaN rra[3].cdp_prep[4].unknown_datapoints = 190
So that looks like it is populating right now.
I deleted the rrd file to start fresh. I remember in the past, that any little changes will confuse the RRD.
And I?ve set the xymon script to run every hour instead of once a day. Unless I run a network by hand, the data will never change except the once a day.
So we?ll see in a while.
From: Japheth Cleaver <cleaver at terabithia.org> Sent: Monday, September 09, 2019 12:45 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Ralph Mitchell' <ralphmitchell at gmail.com> Cc: Stef Coene <stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
You should see four different RRA's (round-robin archives) when you run 'rrdtool info' on the actual RRD file, each being populated as "roll-up" averages of the various data point. For example: ]# rrdtool info clock.rrd
I suspect the issue here is the lack of data in the originating source. It seems like a once-a-day datapoint should at least be made visible in the 2d RRA, but I've not actually tried to configure one with low frequency before. Would you be able to artificially submit that point several times within a 5m period and see if it populates? Keeping --no-cache back on may be beneficial here too.
Regards, -jc
On 9/6/2019 11:01 AM, Root, Paul T wrote: I don?t think the rrd file is working right.
It is there, and the latest numbers are in, but I don?t see any archive numbers populating.
Where can I check for that?
From: Ralph Mitchell <ralphmitchell at gmail.com><mailto:ralphmitchell at gmail.com> Sent: Friday, September 06, 2019 10:07 AM To: Root, Paul T <Paul.Root at CenturyLink.com><mailto:Paul.Root at CenturyLink.com> Cc: Japheth Cleaver <cleaver at terabithia.org><mailto:cleaver at terabithia.org>; Stef Coene <stef.coene at docum.org><mailto:stef.coene at docum.org>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
Could be wrong, but I think the graphs need two or three data points in order to show anything.
Ralph Mitchell
On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: Looking in the rrd file, it doesn't seem like there are the historical data in the file.
The data is only going to change once a day, so I was only calling the script once a day.
I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data.
Does the graphing thing need more frequent data?
-----Original Message----- From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org<mailto:cleaver at terabithia.org>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
"ds[NetAtoD].minimal_heartbeat = 600"
Isn't the minimal_heartbeat setting for telling RRD to discard samples that exist on their own? I think the idea is that it doesn't make sense to calculate values from DERIVE or COUNTER inputs. From the man page for rrdcreate:
<*The "heartbeat" defines the maximum acceptable interval between samples/updates. If the interval between samples is less than "heartbeat", then an average rate is calculated and applied for that interval. If the interval between samples is longer than "heartbeat", then that entire interval is considered "unknown".*>
If you're sampling once per hour, then every sample will exceed the 600 second heartbeat. So you might need to re-create (or rrdtune) your RRD file and increase the mimimal_heartbeat value.
And from the man page for rrdtune:
<*--heartbeat|-h ds-name:heartbeat **modify the heartbeat of a data source. By setting this to a high value the RRD will accept things like one value per day.*>
J
On Tue, 10 Sep 2019 at 07:37, Root, Paul T <Paul.Root at centurylink.com> wrote:
I do see 4 archives:
filename = "ConfigPulls.rrd"
rrd_version = "0003"
step = 300
last_update = 1568044491
ds[NetAtoD].type = "GAUGE"
ds[NetAtoD].minimal_heartbeat = 600
ds[NetAtoD].min = NaN
ds[NetAtoD].max = NaN
ds[NetAtoD].last_ds = "U"
ds[NetAtoD].value = 0.0000000000e+00
ds[NetAtoD].unknown_sec = 291
ds[NetOtoZ1to9].type = "GAUGE"
ds[NetOtoZ1to9].minimal_heartbeat = 600
ds[NetOtoZ1to9].min = NaN
ds[NetOtoZ1to9].max = NaN
ds[NetOtoZ1to9].last_ds = "U"
ds[NetOtoZ1to9].value = 0.0000000000e+00
ds[NetOtoZ1to9].unknown_sec = 291
ds[Other].type = "GAUGE"
ds[Other].minimal_heartbeat = 600
ds[Other].min = NaN
ds[Other].max = NaN
ds[Other].last_ds = "U"
ds[Other].value = 0.0000000000e+00
ds[Other].unknown_sec = 291
ds[InfraGen].type = "GAUGE"
ds[InfraGen].minimal_heartbeat = 600
ds[InfraGen].min = NaN
ds[InfraGen].max = NaN
ds[InfraGen].last_ds = "U"
ds[InfraGen].value = 0.0000000000e+00
ds[InfraGen].unknown_sec = 291
ds[InfraSec].type = "GAUGE"
ds[InfraSec].minimal_heartbeat = 600
ds[InfraSec].min = NaN
ds[InfraSec].max = NaN
ds[InfraSec].last_ds = "U"
ds[InfraSec].value = 0.0000000000e+00
ds[InfraSec].unknown_sec = 291
rra[0].cf = "AVERAGE"
rra[0].rows = 576
rra[0].cur_row = 234
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[0].cdp_prep[2].value = NaN
rra[0].cdp_prep[2].unknown_datapoints = 0
rra[0].cdp_prep[3].value = NaN
rra[0].cdp_prep[3].unknown_datapoints = 0
rra[0].cdp_prep[4].value = NaN
rra[0].cdp_prep[4].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 576
rra[1].cur_row = 337
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 4
rra[1].cdp_prep[1].value = NaN
rra[1].cdp_prep[1].unknown_datapoints = 4
rra[1].cdp_prep[2].value = NaN
rra[1].cdp_prep[2].unknown_datapoints = 4
rra[1].cdp_prep[3].value = NaN
rra[1].cdp_prep[3].unknown_datapoints = 4
rra[1].cdp_prep[4].value = NaN
rra[1].cdp_prep[4].unknown_datapoints = 4
rra[2].cf = "AVERAGE"
rra[2].rows = 576
rra[2].cur_row = 227
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 22
rra[2].cdp_prep[1].value = NaN
rra[2].cdp_prep[1].unknown_datapoints = 22
rra[2].cdp_prep[2].value = NaN
rra[2].cdp_prep[2].unknown_datapoints = 22
rra[2].cdp_prep[3].value = NaN
rra[2].cdp_prep[3].unknown_datapoints = 22
rra[2].cdp_prep[4].value = NaN
rra[2].cdp_prep[4].unknown_datapoints = 22
rra[3].cf = "AVERAGE"
rra[3].rows = 576
rra[3].cur_row = 172
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = NaN
rra[3].cdp_prep[0].unknown_datapoints = 190
rra[3].cdp_prep[1].value = NaN
rra[3].cdp_prep[1].unknown_datapoints = 190
rra[3].cdp_prep[2].value = NaN
rra[3].cdp_prep[2].unknown_datapoints = 190
rra[3].cdp_prep[3].value = NaN
rra[3].cdp_prep[3].unknown_datapoints = 190
rra[3].cdp_prep[4].value = NaN
rra[3].cdp_prep[4].unknown_datapoints = 190
So that looks like it is populating right now.
I deleted the rrd file to start fresh. I remember in the past, that any little changes will confuse the RRD.
And I?ve set the xymon script to run every hour instead of once a day. Unless I run a network by hand, the data will never change except the once a day.
So we?ll see in a while.
*From:* Japheth Cleaver <cleaver at terabithia.org> *Sent:* Monday, September 09, 2019 12:45 PM *To:* Root, Paul T <Paul.Root at CenturyLink.com>; 'Ralph Mitchell' < ralphmitchell at gmail.com> *Cc:* Stef Coene <stef.coene at docum.org>; xymon at xymon.com *Subject:* Re: [Xymon] Graphing in Xymon
You should see four different RRA's (round-robin archives) when you run 'rrdtool info' on the actual RRD file, each being populated as "roll-up" averages of the various data point. For example: ]# rrdtool info clock.rrd
I suspect the issue here is the lack of data in the originating source. It seems like a once-a-day datapoint should at least be made visible in the 2d RRA, but I've not actually tried to configure one with low frequency before. Would you be able to artificially submit that point several times within a 5m period and see if it populates? Keeping --no-cache back on may be beneficial here too.
Regards,
-jc
On 9/6/2019 11:01 AM, Root, Paul T wrote:
I don?t think the rrd file is working right.
It is there, and the latest numbers are in, but I don?t see any archive numbers populating.
Where can I check for that?
*From:* Ralph Mitchell <ralphmitchell at gmail.com> <ralphmitchell at gmail.com> *Sent:* Friday, September 06, 2019 10:07 AM *To:* Root, Paul T <Paul.Root at CenturyLink.com> <Paul.Root at CenturyLink.com> *Cc:* Japheth Cleaver <cleaver at terabithia.org> <cleaver at terabithia.org>; Stef Coene <stef.coene at docum.org> <stef.coene at docum.org>; xymon at xymon.com *Subject:* Re: [Xymon] Graphing in Xymon
Could be wrong, but I think the graphs need two or three data points in order to show anything.
Ralph Mitchell
On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <Paul.Root at centurylink.com> wrote:
Looking in the rrd file, it doesn't seem like there are the historical data in the file.
The data is only going to change once a day, so I was only calling the script once a day.
I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data.
Does the graphing thing need more frequent data?
-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It onlyhas 1 data point at this time, but again. Running rrdtool graph on that file produces a graph
Oh, I never mentioned before. This is CentOS 6 with my ownbuild for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.
Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' < stef.coene at docum.org>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget " http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... " --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Excellent, that?s probably the key.
I?ve made the change, I?m guessing I need a new input before anything can work.
rrdtool tune ConfigPulls.rrd -h NetAtoD:86400 -h NetEtoN:86400 -h NetOtoZ1to9:86400 -h Other:86400 -h InfraGen:86400 -h InfraSec:86400
<?xml version="1.0" encoding="utf-8"?> <!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd"> <!-- Round Robin Database Dump --><rrd> <version> 0003 </version> <step> 300 </step> <!-- Seconds --> <lastupdate> 1568123701 </lastupdate> <!-- 2019-09-10 08:55:01 CDT -->
<ds>
<name> NetAtoD </name>
<type> GAUGE </type>
<minimal_heartbeat> 86400 </minimal_heartbeat>
<min> NaN </min>
<max> NaN </max>
<!-- PDP Status -->
<last_ds> 167 </last_ds>
<value> NaN </value>
<unknown_sec> 1 </unknown_sec>
</ds>Impatient as I am, I manually ran the script again, and then ran the rrdtool graph, and I got data points on the graph. So I should be close now.
Thanks, Paul.
From: Jeremy Laidman <jeremy at laidman.org> Sent: Tuesday, September 10, 2019 8:38 AM To: Root, Paul T <Paul.Root at CenturyLink.com> Cc: Japheth Cleaver <cleaver at terabithia.org>; Ralph Mitchell <ralphmitchell at gmail.com>; xymon at xymon.com Subject: Re: [Xymon] Graphing in Xymon
"ds[NetAtoD].minimal_heartbeat = 600"
Isn't the minimal_heartbeat setting for telling RRD to discard samples that exist on their own? I think the idea is that it doesn't make sense to calculate values from DERIVE or COUNTER inputs. From the man page for rrdcreate:
<The "heartbeat" defines the maximum acceptable interval between samples/updates. If the interval between samples is less than "heartbeat", then an average rate is calculated and applied for that interval. If the interval between samples is longer than "heartbeat", then that entire interval is considered "unknown".>
If you're sampling once per hour, then every sample will exceed the 600 second heartbeat. So you might need to re-create (or rrdtune) your RRD file and increase the mimimal_heartbeat value.
And from the man page for rrdtune:
<--heartbeat|-h ds-name:heartbeat modify the heartbeat of a data source. By setting this to a high value the RRD will accept things like one value per day.>
J
On Tue, 10 Sep 2019 at 07:37, Root, Paul T <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: I do see 4 archives: filename = "ConfigPulls.rrd" rrd_version = "0003" step = 300 last_update = 1568044491 ds[NetAtoD].type = "GAUGE" ds[NetAtoD].minimal_heartbeat = 600 ds[NetAtoD].min = NaN ds[NetAtoD].max = NaN ds[NetAtoD].last_ds = "U" ds[NetAtoD].value = 0.0000000000e+00 ds[NetAtoD].unknown_sec = 291 ds[NetOtoZ1to9].type = "GAUGE" ds[NetOtoZ1to9].minimal_heartbeat = 600 ds[NetOtoZ1to9].min = NaN ds[NetOtoZ1to9].max = NaN ds[NetOtoZ1to9].last_ds = "U" ds[NetOtoZ1to9].value = 0.0000000000e+00 ds[NetOtoZ1to9].unknown_sec = 291 ds[Other].type = "GAUGE" ds[Other].minimal_heartbeat = 600 ds[Other].min = NaN ds[Other].max = NaN ds[Other].last_ds = "U" ds[Other].value = 0.0000000000e+00 ds[Other].unknown_sec = 291 ds[InfraGen].type = "GAUGE" ds[InfraGen].minimal_heartbeat = 600 ds[InfraGen].min = NaN ds[InfraGen].max = NaN ds[InfraGen].last_ds = "U" ds[InfraGen].value = 0.0000000000e+00 ds[InfraGen].unknown_sec = 291 ds[InfraSec].type = "GAUGE" ds[InfraSec].minimal_heartbeat = 600 ds[InfraSec].min = NaN ds[InfraSec].max = NaN ds[InfraSec].last_ds = "U" ds[InfraSec].value = 0.0000000000e+00 ds[InfraSec].unknown_sec = 291 rra[0].cf = "AVERAGE" rra[0].rows = 576 rra[0].cur_row = 234 rra[0].pdp_per_row = 1 rra[0].xff = 5.0000000000e-01 rra[0].cdp_prep[0].value = NaN rra[0].cdp_prep[0].unknown_datapoints = 0 rra[0].cdp_prep[1].value = NaN rra[0].cdp_prep[1].unknown_datapoints = 0 rra[0].cdp_prep[2].value = NaN rra[0].cdp_prep[2].unknown_datapoints = 0 rra[0].cdp_prep[3].value = NaN rra[0].cdp_prep[3].unknown_datapoints = 0 rra[0].cdp_prep[4].value = NaN rra[0].cdp_prep[4].unknown_datapoints = 0 rra[1].cf = "AVERAGE" rra[1].rows = 576 rra[1].cur_row = 337 rra[1].pdp_per_row = 6 rra[1].xff = 5.0000000000e-01 rra[1].cdp_prep[0].value = NaN rra[1].cdp_prep[0].unknown_datapoints = 4 rra[1].cdp_prep[1].value = NaN rra[1].cdp_prep[1].unknown_datapoints = 4 rra[1].cdp_prep[2].value = NaN rra[1].cdp_prep[2].unknown_datapoints = 4 rra[1].cdp_prep[3].value = NaN rra[1].cdp_prep[3].unknown_datapoints = 4 rra[1].cdp_prep[4].value = NaN rra[1].cdp_prep[4].unknown_datapoints = 4 rra[2].cf = "AVERAGE" rra[2].rows = 576 rra[2].cur_row = 227 rra[2].pdp_per_row = 24 rra[2].xff = 5.0000000000e-01 rra[2].cdp_prep[0].value = NaN rra[2].cdp_prep[0].unknown_datapoints = 22 rra[2].cdp_prep[1].value = NaN rra[2].cdp_prep[1].unknown_datapoints = 22 rra[2].cdp_prep[2].value = NaN rra[2].cdp_prep[2].unknown_datapoints = 22 rra[2].cdp_prep[3].value = NaN rra[2].cdp_prep[3].unknown_datapoints = 22 rra[2].cdp_prep[4].value = NaN rra[2].cdp_prep[4].unknown_datapoints = 22 rra[3].cf = "AVERAGE" rra[3].rows = 576 rra[3].cur_row = 172 rra[3].pdp_per_row = 288 rra[3].xff = 5.0000000000e-01 rra[3].cdp_prep[0].value = NaN rra[3].cdp_prep[0].unknown_datapoints = 190 rra[3].cdp_prep[1].value = NaN rra[3].cdp_prep[1].unknown_datapoints = 190 rra[3].cdp_prep[2].value = NaN rra[3].cdp_prep[2].unknown_datapoints = 190 rra[3].cdp_prep[3].value = NaN rra[3].cdp_prep[3].unknown_datapoints = 190 rra[3].cdp_prep[4].value = NaN rra[3].cdp_prep[4].unknown_datapoints = 190
So that looks like it is populating right now.
I deleted the rrd file to start fresh. I remember in the past, that any little changes will confuse the RRD.
And I?ve set the xymon script to run every hour instead of once a day. Unless I run a network by hand, the data will never change except the once a day.
So we?ll see in a while.
From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Monday, September 09, 2019 12:45 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Ralph Mitchell' <ralphmitchell at gmail.com<mailto:ralphmitchell at gmail.com>> Cc: Stef Coene <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
You should see four different RRA's (round-robin archives) when you run 'rrdtool info' on the actual RRD file, each being populated as "roll-up" averages of the various data point. For example: ]# rrdtool info clock.rrd
I suspect the issue here is the lack of data in the originating source. It seems like a once-a-day datapoint should at least be made visible in the 2d RRA, but I've not actually tried to configure one with low frequency before. Would you be able to artificially submit that point several times within a 5m period and see if it populates? Keeping --no-cache back on may be beneficial here too.
Regards, -jc
On 9/6/2019 11:01 AM, Root, Paul T wrote: I don?t think the rrd file is working right.
It is there, and the latest numbers are in, but I don?t see any archive numbers populating.
Where can I check for that?
From: Ralph Mitchell <ralphmitchell at gmail.com><mailto:ralphmitchell at gmail.com> Sent: Friday, September 06, 2019 10:07 AM To: Root, Paul T <Paul.Root at CenturyLink.com><mailto:Paul.Root at CenturyLink.com> Cc: Japheth Cleaver <cleaver at terabithia.org><mailto:cleaver at terabithia.org>; Stef Coene <stef.coene at docum.org><mailto:stef.coene at docum.org>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
Could be wrong, but I think the graphs need two or three data points in order to show anything.
Ralph Mitchell
On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: Looking in the rrd file, it doesn't seem like there are the historical data in the file.
The data is only going to change once a day, so I was only calling the script once a day.
I'm going to run the xymon script every hour to see if that gets the pump primed, and give me more data.
Does the graphing thing need more frequent data?
-----Original Message----- From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Root, Paul T Sent: Friday, September 06, 2019 8:28 AM To: 'Japheth Cleaver' <cleaver at terabithia.org<mailto:cleaver at terabithia.org>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
No difference.
I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 5:19 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
Hmm.
That's definitely odd, but if at least some of the graphs are showing then it's not an overall systemic issue. Have you tried removing the --no-cache from the RRD calls by any chance?
-jc
On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok
$ fuser * rrdctl.2339: 2339 rrdctl.2437: 2437 $ ps -ef |grep 2339 xymon 2339 2323 0 15:09 ? 00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5083 4674 0 15:43 pts/0 00:00:00 grep 2339 $ ps -ef |grep 2437 xymon 2437 2324 0 15:09 ? 00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache root 5086 4674 0 15:43 pts/0 00:00:00 grep 2437 $ ls -la total 19964 drwxr-xr-x. 2 xymon xymon 4096 Sep 5 15:39 . drwxr-xr-x. 12 xymon xymon 4096 Mar 1 2016 .. -rw-rw-r--. 1 xymon xymon 52136 Sep 5 15:43 alert.chk -rw-rw-r--. 1 xymon xymon 96 Sep 5 15:43 alert.chk.sub -rw-r--r--. 1 xymon xymon 24 Sep 5 15:39 ping..status srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2339 srw-rw-rw-. 1 xymon xymon 0 Sep 5 15:09 rrdctl.2437 -rw-r--r--. 1 xymon xymon 22 Oct 19 2018 ssh_8022..status -rw-r--r--. 1 xymon xymon 20366660 Sep 5 15:39 xymond.chk
Selinux is set to permissive.
Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 3:33 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
This is probably a sign of invalid rrdctl socket files, as that's what showgraph is trying to use to communicate with xymond_rrd.
These should be in the TMP/RUN dir on a source compile, named "rrdctl.<pid>", one for each of the currently running xymond_rrd processes (usually two,m one for status, one for data). They'll need to be writable by your apache user at least, and SELinux may require additional permissions for the sendto if you have it enabled and enforcing. If you see lots of stale ones, be sure to remove them -- as showgraph.sh doesn't know what's current, I believe it tries to cycle through all of them, which could end up in timeouts.
Regardless, premature header ending isn't a desired result no matter what the situation. Trying to find all cases of this is a longer term goal for all the CGIs.
HTH, -jc
On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah, I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE... [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SE...
And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph Oh, I never mentioned before. This is CentOS 6 with my own build for 4.3.21. I'm starting to move to the Terabithia repo, but haven't got there yet on the server.Paul.
-----Original Message----- From: Japheth Cleaver <cleaver at terabithia.org<mailto:cleaver at terabithia.org>> Sent: Thursday, September 05, 2019 3:00 PM To: Root, Paul T <Paul.Root at CenturyLink.com<mailto:Paul.Root at CenturyLink.com>>; 'Stef Coene' <stef.coene at docum.org<mailto:stef.coene at docum.org>>; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
A 500 ISE isn't a good sign. Is there anything in the web error logs?
And just to confirm, the file does actually existing in the RRD directory as expected?
Regards, -jc
On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:
$ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser..." --2019-09-05 14:46:58-- http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&ser... Resolving nsmdenvp117.corp.intranet... 151.119.9.117 Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg. But that hasn't translated into a graph.
rrdtool graph ConfigPulls.jpg
--title "NCM Config Pulls"
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE"
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE"
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE"
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE"
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE"
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE"
"LINE2:NetAtoD#00FF00:Net A to D"
"LINE2:NetEtoN#00CCCC:Net E to N"
"LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9"
"LINE2:Other#FF0000:Other"
"LINE2:InfraGen#FFFF00:Infrastructure General"
"LINE2:InfraSec#FFFFCC:Infrastructure Secure"
"COMMENT:\n"-----Original Message----- From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> On Behalf Of Stef Coene Sent: Thursday, September 05, 2019 2:18 PM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] Graphing in Xymon
On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the test and trends pages.
Any help? Then there is a problem generating the RRD, the error is returned as text. But you browser expects a graph, not some text so it displays a broken image symbol.
When yoy right click on one of the broken image, you can say 'open image in new tab'. This will give you the direct URL of the graph. If you do a wget of this URL (don't forget quotes!) on your xymon server, you will download the real error. Just do a vi of the downloaded file to see this error.
Stef
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
participants (5)
-
cleaver＠terabithia.org
-
jeremy＠laidman.org
-
Paul.Root＠CenturyLink.com
-
ralphmitchell＠gmail.com
-
stef.coene＠docum.org