Strange rrd problem
Hi,
I've set up a hobbit server to monitor a couple of systems.
However the network test graphs from exactly one system are looking quite strange:
http://www.westend.com/pop3-1.png http://www.westend.com/pop3-2.png (zoomed in view)
According to this graph pop3 is answering always within zero seconds (I can't imagine this). And the response time of pop3s is alternating in such specific way that rrd draws triangles.
I've already deleted the rrd files, but the effect remains the same.
The system is running Debian 4.0 (librrd2 version 1.2.15-0.3). Has anybody seen such strange effects before? All other systems and even other services on the same system are looking reasonable.
Ciao, Thomas
Thomas Kähn WESTEND GmbH | Internet-Business-Provider Technik CISCO Systems Partner - Authorized Reseller Im Süsterfeld 6 Tel 0241/701333-18 tk at westend.com D-52072 Aachen Fax 0241/911879
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608 Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
One reason for the flat always-zero pop3 graph could be that it responds faster than Hobbit monitors. If the response time is less than 10 ms, then Hobbit will report it as zero - look at the pop3 status page and see the "Seconds: x.xx" which is the data that Hobbit uses for the graph.
As for the diamond-shaped graph - yes, this is quite puzzling. I've had a couple of other reports (see e.g. this thread: http://www.hobbitmon.com/hobbiton/2005/10/msg00332.html). We haven't been able to figure out why this happens, the data that goes into the RRD file look sane.
Regards, Henrik
On Fri, Jul 13, 2007 at 09:01:10AM +0200, Thomas Kaehn wrote:
Hi,
I've set up a hobbit server to monitor a couple of systems.
However the network test graphs from exactly one system are looking quite strange:
http://www.westend.com/pop3-1.png http://www.westend.com/pop3-2.png (zoomed in view)
According to this graph pop3 is answering always within zero seconds (I can't imagine this). And the response time of pop3s is alternating in such specific way that rrd draws triangles.
I've already deleted the rrd files, but the effect remains the same.
The system is running Debian 4.0 (librrd2 version 1.2.15-0.3). Has anybody seen such strange effects before? All other systems and even other services on the same system are looking reasonable.
Ciao, Thomas
Thomas Kähn WESTEND GmbH | Internet-Business-Provider Technik CISCO Systems Partner - Authorized Reseller Im Süsterfeld 6 Tel 0241/701333-18 tk at westend.com D-52072 Aachen Fax 0241/911879
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608 Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Henrik Storner
Hi Henrik,
On Fri, Jul 13, 2007 at 12:49:22PM +0200, Henrik Stoerner wrote:
One reason for the flat always-zero pop3 graph could be that it responds faster than Hobbit monitors. If the response time is less than 10 ms, then Hobbit will report it as zero - look at the pop3 status page and see the "Seconds: x.xx" which is the data that Hobbit uses for the graph.
OK, this would explain why zero is reported.
As for the diamond-shaped graph - yes, this is quite puzzling. I've had a couple of other reports (see e.g. this thread: http://www.hobbitmon.com/hobbiton/2005/10/msg00332.html). We haven't been able to figure out why this happens, the data that goes into the RRD file look sane.
The data in the rrd file looks strange however when dumped using rrdtool. So every other value is different.
Ciao, Thomas
<!-- 2007-07-13 05:55:00 CEST / 1184298900 --> <row><v> 3.0100000000e+00 </v></row> <!-- 2007-07-13 06:00:00 CEST / 1184299200 --> <row><v> 3.0100000000e+00 </v></row> <!-- 2007-07-13 06:05:00 CEST / 1184299500 --> <row><v> 2.6000000000e+00 </v></row> <!-- 2007-07-13 06:10:00 CEST / 1184299800 --> <row><v> 4.3000000000e-01 </v></row> <!-- 2007-07-13 06:15:00 CEST / 1184300100 --> <row><v> 2.6300000000e+00 </v></row> <!-- 2007-07-13 06:20:00 CEST / 1184300400 --> <row><v> 4.1000000000e-01 </v></row> <!-- 2007-07-13 06:25:00 CEST / 1184300700 --> <row><v> 2.6500000000e+00 </v></row> <!-- 2007-07-13 06:30:00 CEST / 1184301000 --> <row><v> 3.9000000000e-01 </v></row> <!-- 2007-07-13 06:35:00 CEST / 1184301300 --> <row><v> 2.6700000000e+00 </v></row> <!-- 2007-07-13 06:40:00 CEST / 1184301600 --> <row><v> 3.7000000000e-01 </v></row> <!-- 2007-07-13 06:45:00 CEST / 1184301900 --> <row><v> 2.6900000000e+00 </v></row> <!-- 2007-07-13 06:50:00 CEST / 1184302200 --> <row><v> 3.5000000000e-01 </v></row> <!-- 2007-07-13 06:55:00 CEST / 1184302500 --> <row><v> 2.7100000000e+00 </v></row> <!-- 2007-07-13 07:00:00 CEST / 1184302800 --> <row><v> 3.3000000000e-01 </v></row> <!-- 2007-07-13 07:05:00 CEST / 1184303100 --> <row><v> 2.7300000000e+00 </v></row> <!-- 2007-07-13 07:10:00 CEST / 1184303400 --> <row><v> 3.0900000000e-01 </v></row> <!-- 2007-07-13 07:15:00 CEST / 1184303700 --> <row><v> 2.7408666667e+00 </v></row> <!-- 2007-07-13 07:20:00 CEST / 1184304000 --> <row><v> 2.9000000000e-01 </v></row> <!-- 2007-07-13 07:25:00 CEST / 1184304300 --> <row><v> 2.7700000000e+00 </v></row> <!-- 2007-07-13 07:30:00 CEST / 1184304600 --> <row><v> 2.6000000000e-01 </v></row> <!-- 2007-07-13 07:35:00 CEST / 1184304900 --> <row><v> 2.7900000000e+00 </v></row> <!-- 2007-07-13 07:40:00 CEST / 1184305200 --> <row><v> 2.4000000000e-01 </v></row> <!-- 2007-07-13 07:45:00 CEST / 1184305500 --> <row><v> 2.8200000000e+00 </v></row> <!-- 2007-07-13 07:50:00 CEST / 1184305800 --> <row><v> 2.2000000000e-01 </v></row> <!-- 2007-07-13 07:55:00 CEST / 1184306100 --> <row><v> 2.8400000000e+00 </v></row> <!-- 2007-07-13 08:00:00 CEST / 1184306400 --> <row><v> 2.0000000000e-01 </v></row> <!-- 2007-07-13 08:05:00 CEST / 1184306700 --> <row><v> 2.8600000000e+00 </v></row> <!-- 2007-07-13 08:10:00 CEST / 1184307000 --> <row><v> 1.8000000000e-01 </v></row>
-- Thomas Kähn WESTEND GmbH | Internet-Business-Provider Technik CISCO Systems Partner - Authorized Reseller Im Süsterfeld 6 Tel 0241/701333-18 tk at westend.com D-52072 Aachen Fax 0241/911879
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608 Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
On Friday 13 July 2007 07:35, Thomas Kaehn wrote:
On Fri, Jul 13, 2007 at 12:49:22PM +0200, Henrik Stoerner wrote:
As for the diamond-shaped graph - yes, this is quite puzzling. I've had a couple of other reports (see e.g. this thread: http://www.hobbitmon.com/hobbiton/2005/10/msg00332.html). We haven't been able to figure out why this happens, the data that goes into the RRD file look sane.
The data in the rrd file looks strange however when dumped using rrdtool. So every other value is different.
Would this somehow be linked to the ping graphs I see that form 'ramps' ? Does anyone else see these 'ramps' ? Most of my host ping times show the same pattern. So this would define the problem to be in Hobbit, or in my network ;) The BigBrother server, running in parallel with Hobbit, doesn't show this pattern. I am using the hobbitping tool. The small home hobbit server, I run, doesn't show this pattern 'ramp' pattern.
You can see a sample graph at, http://jentoo.homedns.org:8080/pub/download/pingtimes.png
Thanks, ~Steve
Hi,
On Fri, Jul 13, 2007 at 07:55:44AM -0400, s_aiello at comcast.net wrote:
On Friday 13 July 2007 07:35, Thomas Kaehn wrote:
On Fri, Jul 13, 2007 at 12:49:22PM +0200, Henrik Stoerner wrote:
As for the diamond-shaped graph - yes, this is quite puzzling. I've had a couple of other reports (see e.g. this thread: http://www.hobbitmon.com/hobbiton/2005/10/msg00332.html). We haven't been able to figure out why this happens, the data that goes into the RRD file look sane.
The data in the rrd file looks strange however when dumped using rrdtool. So every other value is different.
Would this somehow be linked to the ping graphs I see that form 'ramps' ? Does anyone else see these 'ramps' ? Most of my host ping times show the same pattern. So this would define the problem to be in Hobbit, or in my network ;) The BigBrother server, running in parallel with Hobbit, doesn't show this pattern. I am using the hobbitping tool. The small home hobbit server, I run, doesn't show this pattern 'ramp' pattern.
You can see a sample graph at, http://jentoo.homedns.org:8080/pub/download/pingtimes.png
this could be a similar problem. I've seen that the bbtest graph corresponds to the network tests showing a wrong graph. Maybe the problem is caused by bbtest-net?
Please compare:
http://www.westend.com/bbtest.png
http://www.westend.com/pop.png
Is there any possibility to debug this problem?
Ciao, Thomas
Thomas Kähn WESTEND GmbH | Internet-Business-Provider Technik CISCO Systems Partner - Authorized Reseller Im Süsterfeld 6 Tel 0241/701333-18 tk at westend.com D-52072 Aachen Fax 0241/911879
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608 Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
Hi,
On Mon, Jul 16, 2007 at 01:16:45PM +0200, Thomas Kaehn wrote:
On Fri, Jul 13, 2007 at 07:55:44AM -0400, s_aiello at comcast.net wrote:
On Friday 13 July 2007 07:35, Thomas Kaehn wrote:
On Fri, Jul 13, 2007 at 12:49:22PM +0200, Henrik Stoerner wrote:
As for the diamond-shaped graph - yes, this is quite puzzling. I've had a couple of other reports (see e.g. this thread: http://www.hobbitmon.com/hobbiton/2005/10/msg00332.html). We haven't been able to figure out why this happens, the data that goes into the RRD file look sane.
this could be a similar problem. I've seen that the bbtest graph corresponds to the network tests showing a wrong graph. Maybe the problem is caused by bbtest-net?
Please compare: http://www.westend.com/bbtest.png http://www.westend.com/pop.png
BTW, I've moved the hobbit installation to a different system and the graphs look normal now. Neither bbtest nor the different network tests show the "diamonds" despite Hobbit and rrd versions are the same.
Ciao, Thomas
Thomas Kähn WESTEND GmbH | Internet-Business-Provider Technik CISCO Systems Partner - Authorized Reseller Im Süsterfeld 6 Tel 0241/701333-18 tk at westend.com D-52072 Aachen Fax 0241/911879
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608 Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
On Friday 13 July 2007, Thomas Kaehn wrote:
Hi,
I've set up a hobbit server to monitor a couple of systems.
However the network test graphs from exactly one system are looking quite strange:
http://www.westend.com/pop3-1.png http://www.westend.com/pop3-2.png (zoomed in view)
According to this graph pop3 is answering always within zero seconds (I can't imagine this). And the response time of pop3s is alternating in such specific way that rrd draws triangles.
I've already deleted the rrd files, but the effect remains the same.
The system is running Debian 4.0 (librrd2 version 1.2.15-0.3). Has anybody seen such strange effects before? All other systems and even other services on the same system are looking reasonable. Today, I noticed the same problem. I have 2 hobbit servers: AIX with rrdtool-1.2.10-2 and a linux box with 1.2.15-0.3ubuntu1. The AIX box is ok, the linux box shows the same trends. I only see this in the http graphs. Not in the ssh graphs.
Maybe a problem with version 1.2.15 ?
Stef
participants (4)
-
henrik＠hswn.dk
-
s_aiello＠comcast.net
-
stef.coene＠docum.org
-
tk＠westend.com