disk graph page limits total file systems
I'm having some trouble with the disk graph pages for hosts that have numerous file systems reporting in. The limit seems to be 85. There are 5 file systems displayed on each graph and there are 17 graphs max on this page. If I view the trends page for those host, all of the missing filesystems are graphed. 85 seems pretty arbitrary.
I've gone through and cleaned up the RRD files from $XYMON/data/rrd/$HOST that are "stale". Cleaning some of these out, which would otherwise not be listed, permit the filesystems that were trimmed to then be listed on the graph page. Still, not all of the filesystems make it on the graphs.
I searched the mail archive, and didn't find a solutions. I found a discussion of changing the [disk] section to [disk::10] for 10 filesystems per graph... that prevented the graphs from being generated at all. My thinking was, perhaps if i increase the number of filesystems per graph, 17 graphs would be enough for this particular system.
In any event, I'm wondering if this is a bug or a configurable. It actually works fine in a previous version of xymon. (v4.2.2 works, v.4.3.24). Thanks!
Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com
On Tue, May 17, 2016 11:26 am, EDSchminke at Hormel.com wrote:
I'm having some trouble with the disk graph pages for hosts that have numerous file systems reporting in. The limit seems to be 85. There are 5 file systems displayed on each graph and there are 17 graphs max on this page. If I view the trends page for those host, all of the missing filesystems are graphed. 85 seems pretty arbitrary.
I've gone through and cleaned up the RRD files from $XYMON/data/rrd/$HOST that are "stale". Cleaning some of these out, which would otherwise not be listed, permit the filesystems that were trimmed to then be listed on the graph page. Still, not all of the filesystems make it on the graphs.
I searched the mail archive, and didn't find a solutions. I found a discussion of changing the [disk] section to [disk::10] for 10 filesystems per graph... that prevented the graphs from being generated at all. My thinking was, perhaps if i increase the number of filesystems per graph, 17 graphs would be enough for this particular system.
In any event, I'm wondering if this is a bug or a configurable. It actually works fine in a previous version of xymon. (v4.2.2 works, v.4.3.24). Thanks!
Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com
Erik,
85 isn't an intentional hard limit here. I've been scanning through the showgraph code and it seems like the reallocation should be able to continue as needed (whether not having a hard limit at all is a good idea is a separate question...). There's a reference to 16 arguments to rrd_graph, however that's a per-graph value and I don't believe it would affect the number here.
A couple of next steps:
- Can you increase to disk::6 or 7 and see if there's a point where the parsing of that number breaks?
- Does it constantly die at the same partition being graphed?
- Are there any errors coming out in /logs/ or the httpd error log, or core files left?
- Are there any unusual file conditions in that directory?
I'd definitely suggest upgrading to a new version for security purposes, but I don't think any fixes addressing this area specifically are present.
There were a lot of changes between 4.2.2 and 4.3.x, so it's hard to say exactly what might be contributing there.
HTH, -jc
I could try upgrading. At this point, it would be relatively easy since this particular deployment isn't "in production" yet.
I, too, thought 85 was too arbitrary to have been an imposed, hard-coded limit. Especially given that the graphs are generated and displayed on the trends page.
It does not always break at the same partition-- or filesystem. But it always breaks at the 85th partition. If I list the rrd directory for a host (ls $XYMON/data/rrd/$HOST/disk* | sed 's/[.]rrd//' | sort) the last filesystem on the last graph will always be the 85th line of output. If I delete an rrd file (for a filesystem i really don't care about) and look at the page again.... 85th line. Everytime.
I've found no core files and there are no errors to be found in any of the xymon or httpd logs. I'm willing to turn on more verbose logging to the httpd server, just let me know how high you'd like me to turn it up.
I'm also willing to attempt a reconfiguration to show more filesystems per graph, but I'm not 100% clear on how to go about that. What I thought was the correct way to do it was unsuccessful.
From my graphs.cfg: [disk] FNPATTERN ^disk(.*).rrd TITLE Disk Utilization YAXIS % Full DEF:p at RRDIDX@=@RRDFN@:pct:AVERAGE LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@ -u 100 -l 0 GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur) GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max) GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min) GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n
Do I simply change "disk" to "disk::10"? ...because if I do that, no graphs for disk are generated... i.e.:
[disk::10] FNPATTERN ^disk(.*).rrd TITLE Disk Utilization YAXIS % Full DEF:p at RRDIDX@=@RRDFN@:pct:AVERAGE LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@ -u 100 -l 0 GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur) GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max) GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min) GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n
If I'm doing that wrong, let me know. Thanks.
Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com
From: "J.C. Cleaver" <cleaver at terabithia.org> To: EDSchminke at Hormel.com Cc: "Xymon Mailing List" <xymon at xymon.com> Date: 05/17/2016 04:53 PM Subject: Re: [Xymon] disk graph page limits total file systems
Erik,
85 isn't an intentional hard limit here. I've been scanning through the showgraph code and it seems like the reallocation should be able to continue as needed (whether not having a hard limit at all is a good idea is a separate question...). There's a reference to 16 arguments to rrd_graph, however that's a per-graph value and I don't believe it would affect the number here.
A couple of next steps:
- Can you increase to disk::6 or 7 and see if there's a point where the parsing of that number breaks?
- Does it constantly die at the same partition being graphed?
- Are there any errors coming out in /logs/ or the httpd error log, or core files left?
- Are there any unusual file conditions in that directory?
I'd definitely suggest upgrading to a new version for security purposes, but I don't think any fixes addressing this area specifically are present.
There were a lot of changes between 4.2.2 and 4.3.x, so it's hard to say exactly what might be contributing there.
HTH, -jc
update: i upgraded v4.3.24 to v4.3.27. this had no effect on my problem. 85 filesystem "limit" still exists.
Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com
From: "J.C. Cleaver" <cleaver at terabithia.org> To: EDSchminke at Hormel.com Cc: "Xymon Mailing List" <xymon at xymon.com> Date: 05/17/2016 04:53 PM Subject: Re: [Xymon] disk graph page limits total file systems
On Tue, May 17, 2016 11:26 am, EDSchminke at Hormel.com wrote:
I'm having some trouble with the disk graph pages for hosts that have numerous file systems reporting in. The limit seems to be 85. There are 5 file systems displayed on each graph and there are 17 graphs max on this page. If I view the trends page for those host, all of the missing filesystems are graphed. 85 seems pretty arbitrary.
I've gone through and cleaned up the RRD files from $XYMON/data/rrd/$HOST that are "stale". Cleaning some of these out, which would otherwise not be listed, permit the filesystems that were trimmed to then be listed on the graph page. Still, not all of the filesystems make it on the graphs.
I searched the mail archive, and didn't find a solutions. I found a discussion of changing the [disk] section to [disk::10] for 10
filesystems
per graph... that prevented the graphs from being generated at all. My thinking was, perhaps if i increase the number of filesystems per graph, 17 graphs would be enough for this particular system.
In any event, I'm wondering if this is a bug or a configurable. It actually works fine in a previous version of xymon. (v4.2.2 works, v.4.3.24). Thanks!
Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com
Erik,
85 isn't an intentional hard limit here. I've been scanning through the showgraph code and it seems like the reallocation should be able to continue as needed (whether not having a hard limit at all is a good idea is a separate question...). There's a reference to 16 arguments to rrd_graph, however that's a per-graph value and I don't believe it would affect the number here.
A couple of next steps:
- Can you increase to disk::6 or 7 and see if there's a point where the parsing of that number breaks?
- Does it constantly die at the same partition being graphed?
- Are there any errors coming out in /logs/ or the httpd error log, or core files left?
- Are there any unusual file conditions in that directory?
I'd definitely suggest upgrading to a new version for security purposes, but I don't think any fixes addressing this area specifically are present.
There were a lot of changes between 4.2.2 and 4.3.x, so it's hard to say exactly what might be contributing there.
HTH, -jc
participants (2)
-
cleaver＠terabithia.org
-
EDSchminke＠Hormel.com