I am able to give a little better update watching the .rrd files.

It always happens 1 hour apart for three times. Xymon does not update the .rrd files for procs, users and la.

For example.

14:50 no update graph is blank between 14:45 and 14:55 15:50 no update graph is blank between 15:45 and 15:55 16:40 no update graph is blank between 16:45 and 16:55 Watching the time stamps on the files, I saw the following:

On procs.rrd, users.rrd and la.rrd at 16:46 all files were time stamped including the rest of the .rrd files. At 16:51 NO time stamp on procs.rrd, users.rrd, and la.rrd. All other .rrd files were time stamped with 16:51 At 16:56 all files show proper time stamp again.

This is happening at different times on different machines, however when I see two missing data points one hour apart, I can set my watch and see the third missing data point one hour later.

Since this is still happening after I added the --no-cache flag, I am still waiting to see if I added the flag properly since it does not seem to make a difference.

Also,

Here are the packages for rrd that were installed.

rrdtool-1.2.23-1.el5.rf rrdtool-devel-1.2.23-1.el5.rf perl-rrdtool-1.2.23-1.el5.rf

Since I see that the three rrd files are not getting timestamped for the one time, I would assume that it is something within Xymon. Again, I do see any messages in the log files for either the client or server.

Thank you, Tom

-----Original Message----- From: Alan Sparks [mailto:asparks at doublesparks.net] Sent: Friday, November 13, 2009 3:29 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Question about using no-cache on rrd for xymon 4.3.0.0.beta-2

Stewart, Tom L. wrote:

So it looks like it is in effect, but looking at the tmp file I still see the following:

srw-rw-rw- 1 xymon xymon 0 Nov 13 14:26 rrdctl.7672

srw-rw-rw- 1 xymon xymon 0 Nov 13 14:26 rrdctl.7681

When I stop and restart xymon I still get messages such as:

rrd-status.log:2009-11-13 14:26:09 Cache flush completed

rrd-status.log:2009-11-13 14:26:18 Peer not up, flushing message queue

So my question is have I placed the -no-cache in the wrong sequence on the startup command or is that been taken out of beta-2?

The more poignant question is, are you still seeing the update lags? And are the timestamps on the actual RRD files changing (not the rrd channel control files, as you show)? The option does exist in xymon-4.3.0-beta2 source. -Alan

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

I had the same problems as you with missing/gappy data, however –nocache seemed to work just fine. I haven’t had any issues since I started using it. Are you sure you’re calling it correctly?

CMD hobbitd_channel --channel=data --log=/dev/null hobbitd_rrd --no-cache --rrddir=/var/lib/xymon/rrd

CMD hobbitd_channel --channel=status --log=/dev/null hobbitd_rrd --no-cache --rrddir=/var/lib/xymon/rrd

… are what my rrddata and rrdstatus sections are using, respectively

Regards,

-jc

From: Stewart, Tom L. [mailto:Tom.Stewart at landsend.com] Sent: Thursday, November 19, 2009 7:48 AM To: hobbit at hswn.dk Subject: RE: [hobbit] Question about using no-cache on rrd for xymon 4.3.0.0.beta-2

It has been almost a week and no-one has answered my question about turning off caching in the latest xymon code

I checked the source code and my –no-cache flag seems to be correct, but the caching is continuing and is causing holes ( no update for a five minute period) in the rrd files. I also found where –no-cache is set within do_rrd.c and hobbitd_rrd.c, so I am assuming that there is a bug here that does not turn off caching using the –no-cache flag.

So for a temporary fix, can I just set:

do_rrd.c:int use_rrd_cache = 1; /* Use the cache by default */

to

do_rrd.c:int use_rrd_cache = 0; /* Don’t use the cache by default */

recompile and move the binary over or would I be wasting my time as the caching would still continue?

Tom

From: Stewart, Tom L. [mailto:Tom.Stewart at landsend.com] Sent: Friday, November 13, 2009 3:19 PM To: hobbit at hswn.dk Subject: [hobbit] Question about using no-cache on rrd for xymon 4.3.0.0.beta-2

We are having issues with rrd leaving 5-10 minute intervals of no data for items such as cpu load on various systems. Most of the time it happens three times in a row like at 2, 3 and 4 pm. I am not finding anything is the logs on either the client or server. Some googling indicated that the issue may go away by using the no-cache option for RRD. I have added it to the hobbitlaunch.cfg as such:

hobbitlaunch.cfg: CMD hobbitd_channel --channel=status --log=$BBSERVERLOGS/rrd-status.log hobbitd_rrd --no-cache --extra-tests=cpucisco,ifaload,ifload,vload,wphlstat,wperrors --extra-script=/home/xymon/server/ext/extra-rrd.pl --rrddir=$BBVAR/rrd

hobbitlaunch.cfg: CMD hobbitd_channel --channel=data --log=$BBSERVERLOGS/rrd-data.log hobbitd_rrd --no-cache --extra-tests=mpstat,zonestat --extra-script=/home/xymon/server/ext/rrd_data.pl --rrddir=$BBVAR/rrd

This is on a 32 bit red hat system and when I do a ps –ef | grep rrd I show the following:

xymon 7635 7599 0 14:26 ? 00:00:01 hobbitd_channel --channel=status --log=/home/xymon/logs/rrd-status.log hobbitd_rrd --no-cache --extra-tests=cpucisco,ifaload,ifload,vload,wphlstat,wperrors --extra-script=/home/xymon/server/ext/extra-rrd.pl --rrddir=/home/xymon/data/rrd

xymon 7636 7599 0 14:26 ? 00:00:00 hobbitd_channel --channel=data --log=/home/xymon/logs/rrd-data.log hobbitd_rrd --no-cache --extra-tests=mpstat,zonestat --extra-script=/home/xymon/server/ext/rrd_data.pl --rrddir=/home/xymon/data/rrd

xymon 7672 7635 0 14:26 ? 00:00:07 hobbitd_rrd --no-cache --extra-tests=cpucisco,ifaload,ifload,vload,wphlstat,wperrors --extra-script=/home/xymon/server/ext/extra-rrd.pl --rrddir=/home/xymon/data/rrd

xymon 7681 7636 0 14:26 ? 00:00:03 hobbitd_rrd --no-cache --extra-tests=mpstat,zonestat --extra-script=/home/xymon/server/ext/rrd_data.pl --rrddir=/home/xymon/data/rrd

So it looks like it is in effect, but looking at the tmp file I still see the following:

srw-rw-rw- 1 xymon xymon 0 Nov 13 14:26 rrdctl.7672

srw-rw-rw- 1 xymon xymon 0 Nov 13 14:26 rrdctl.7681

When I stop and restart xymon I still get messages such as:

rrd-status.log:2009-11-13 14:26:09 Cache flush completed

rrd-status.log:2009-11-13 14:26:18 Peer not up, flushing message queue

So my question is have I placed the –no-cache in the wrong sequence on the startup command or is that been taken out of beta-2?

Thank you,

Tom

Well, I have tried the --no-cache in exactly the same place as you and it did not help

I found this when I recompiled and set the use_rrd_cache = 0; After recompiling, setting use_rrd_cache to 0 still started caching and is still giving me holes.

gcc -g -O2 -Wall -Wno-unused -D_REENTRANT -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DLINUX -I. -I/home/hobbit_org/source/xymon-4.3.0-beta2/include -I/usr/include -I/usr/include -I/usr/include -c -o do_rrd.o do_rrd.c rrd/do_iostat.c: In function âdo_iostatdisk_rrdâ: rrd/do_iostat.c:74: warning: âidâ may be used uninitialized in this function gcc -o hobbitd_rrd -Wl,--rpath,/usr/lib -Wl,--rpath,/usr/lib -Wl,--rpath,/usr/lib -Wl,--rpath,/usr/lib hobbitd_rrd.o

I am compling on a 32-bin linux RHEL5 system, can this be the source of my issues?

Tom

-----Original Message----- From: Dominique Frise [mailto:dominique.frise at unil.ch] Sent: Friday, November 20, 2009 12:24 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Question about using no-cache on rrd for xymon 4.3.0.0.beta-2

Stewart, Tom L. wrote:

It has been almost a week and no-one has answered my question about turning off caching in the latest xymon code

I checked the source code and my -no-cache flag seems to be correct, but the caching is continuing and is causing holes ( no update for a five minute period) in the rrd files. I also found where -no-cache is set within do_rrd.c and hobbitd_rrd.c, so I am assuming that there is a bug here that does not turn off caching using the -no-cache flag.

So for a temporary fix, can I just set:

do_rrd.c:int use_rrd_cache = 1; /* Use the cache by default */

to

do_rrd.c:int use_rrd_cache = 0; /* Don't use the cache by default */

recompile and move the binary over or would I be wasting my time as the caching would still continue?

Tom

---

*From:* Stewart, Tom L. [mailto:Tom.Stewart at landsend.com] *Sent:* Friday, November 13, 2009 3:19 PM *To:* hobbit at hswn.dk *Subject:* [hobbit] Question about using no-cache on rrd for xymon 4.3.0.0.beta-2

We are having issues with rrd leaving 5-10 minute intervals of no data for items such as cpu load on various systems. Most of the time it happens three times in a row like at 2, 3 and 4 pm. I am not finding anything is the logs on either the client or server. Some googling indicated that the issue may go away by using the no-cache option for RRD. I have added it to the hobbitlaunch.cfg as such:

hobbitlaunch.cfg: CMD hobbitd_channel --channel=status --log=$BBSERVERLOGS/rrd-status.log hobbitd_rrd --no-cache --extra-tests=cpucisco,ifaload,ifload,vload,wphlstat,wperrors --extra-script=/home/xymon/server/ext/extra-rrd.pl --rrddir=$BBVAR/rrd

hobbitlaunch.cfg: CMD hobbitd_channel --channel=data
--log=$BBSERVERLOGS/rrd-data.log hobbitd_rrd --no-cache --extra-tests=mpstat,zonestat
--extra-script=/home/xymon/server/ext/rrd_data.pl --rrddir=$BBVAR/rrd

This is on a 32 bit red hat system and when I do a ps -ef | grep rrd I show the following:

xymon 7635 7599 0 14:26 ? 00:00:01 hobbitd_channel --channel=status --log=/home/xymon/logs/rrd-status.log hobbitd_rrd --no-cache --extra-tests=cpucisco,ifaload,ifload,vload,wphlstat,wperrors --extra-script=/home/xymon/server/ext/extra-rrd.pl --rrddir=/home/xymon/data/rrd

xymon 7636 7599 0 14:26 ? 00:00:00 hobbitd_channel --channel=data --log=/home/xymon/logs/rrd-data.log hobbitd_rrd --no-cache --extra-tests=mpstat,zonestat --extra-script=/home/xymon/server/ext/rrd_data.pl --rrddir=/home/xymon/data/rrd

xymon 7672 7635 0 14:26 ? 00:00:07 hobbitd_rrd --no-cache --extra-tests=cpucisco,ifaload,ifload,vload,wphlstat,wperrors --extra-script=/home/xymon/server/ext/extra-rrd.pl --rrddir=/home/xymon/data/rrd

xymon 7681 7636 0 14:26 ? 00:00:03 hobbitd_rrd --no-cache --extra-tests=mpstat,zonestat --extra-script=/home/xymon/server/ext/rrd_data.pl --rrddir=/home/xymon/data/rrd

So it looks like it is in effect, but looking at the tmp file I still see the following:

srw-rw-rw- 1 xymon xymon 0 Nov 13 14:26 rrdctl.7672

srw-rw-rw- 1 xymon xymon 0 Nov 13 14:26 rrdctl.7681

When I stop and restart xymon I still get messages such as:

rrd-status.log:2009-11-13 14:26:09 Cache flush completed

rrd-status.log:2009-11-13 14:26:18 Peer not up, flushing message queue

So my question is have I placed the -no-cache in the wrong sequence on the startup command or is that been taken out of beta-2?

Thank you,

Tom

Hi,

--no-cache together with Xymon 4.3.0-beta2 works for us.

We use it for status and channel like this:

hobbitlaunch.cfg: CMD hobbitd_channel --channel=status --log=$BBSERVERLOGS/rrd-status.log hobbitd_rrd --no-cache --extra-tests=zfs --extra-script=/soft/pub/BB/hobbit/server/ext/rrd-status.sh --rrddir=$BBVAR/rrd hobbitlaunch.cfg: CMD hobbitd_channel --channel=data --log=$BBSERVERLOGS/rrd-data.log hobbitd_rrd --no-cache --extra-tests=zone --extra-script=/soft/pub/BB/hobbit/server/ext/rrd-data.sh --rrddir=$BBVAR/rrd

Dominique

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk