I'm getting the error "could not lock RRD"
Hi I've just installed hobbit 4.2.0 on an archlinux box with rrdtool version 1.2.23-1. Looking at the logs I'm getting the following error every 30 seconds or so as it tries (I assume) to create the graphs:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
I've googled and searched the archives but can't find any other mention of this error. Can anyone suggest a solution?
Thanks
Robin
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Well, moving the log directory to the local disk stopped the error messages but I'm still getting empty graphs.
Dean, the requirements show PCRE, RRDtool, libpng, OpenSSL and OpenLDAP libraries. I've got all of those:
(from ./configure)
Checking for RRDtool ... test-rrd.c: In function 'main': test-rrd.c:30: error: too few arguments to function 'rrd_graph' make: *** [test-compile] Error 1 Not RRDtool 1.0.x, checking for 1.2.x Found RRDtool include files in /usr/include Found RRDtool libraries in /usr/lib Linking RRD with PNG library: -L/usr/lib -lpng
Checking for PCRE ... Found PCRE include files in /usr/include Found PCRE libraries in /usr/lib
Checking for OpenSSL ... Found OpenSSL include files in /usr/include Found OpenSSL libraries in /usr/lib
Any more ideas?
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Got it working! I moved the data directory to the local disk and deleted the contents. After a while it all started working.
rrd obviously doesn't like nfs shares.
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Well, moving the log directory to the local disk stopped the error messages but I'm still getting empty graphs.
Dean, the requirements show PCRE, RRDtool, libpng, OpenSSL and OpenLDAP libraries. I've got all of those:
(from ./configure)
Checking for RRDtool ... test-rrd.c: In function 'main': test-rrd.c:30: error: too few arguments to function 'rrd_graph' make: *** [test-compile] Error 1 Not RRDtool 1.0.x, checking for 1.2.x Found RRDtool include files in /usr/include Found RRDtool libraries in /usr/lib Linking RRD with PNG library: -L/usr/lib -lpng
Checking for PCRE ... Found PCRE include files in /usr/include Found PCRE libraries in /usr/lib
Checking for OpenSSL ... Found OpenSSL include files in /usr/include Found OpenSSL libraries in /usr/lib
Any more ideas?
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
I am running my rrd files on an NFS share, having no problems. Have also ran several hobbit servers with the application binaries housed on NFS mounts; the app binaries & rrd files on NFS mounts; and just the rrd files on NFS mounts. Have been running Hobbit server with both the app binaries and rrd files on an NFS mount, for more than a year now. Unless there are some NFS performance issues on your network, housing those files on an NFS mount isn't the problem.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Sunday, June 24, 2007 3:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Got it working! I moved the data directory to the local disk and deleted the contents. After a while it all started working.
rrd obviously doesn't like nfs shares.
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Well, moving the log directory to the local disk stopped the error messages but I'm still getting empty graphs.
Dean, the requirements show PCRE, RRDtool, libpng, OpenSSL and OpenLDAP libraries. I've got all of those:
(from ./configure)
Checking for RRDtool ... test-rrd.c: In function 'main': test-rrd.c:30: error: too few arguments to function 'rrd_graph' make: *** [test-compile] Error 1 Not RRDtool 1.0.x, checking for 1.2.x Found RRDtool include files in /usr/include Found RRDtool libraries in /usr/lib Linking RRD with PNG library: -L/usr/lib -lpng
Checking for PCRE ... Found PCRE include files in /usr/include Found PCRE libraries in /usr/lib
Checking for OpenSSL ... Found OpenSSL include files in /usr/include Found OpenSSL libraries in /usr/lib
Any more ideas?
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Well, something fixed it and I'd rather not break again it while fiddling around to try to find out what it was!
The nfs config is straight out of the box, nothing special and everything else runs ok. Maybe it was something corrupt in the data files and clearing them after moving the directory cleared the problem.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
I am running my rrd files on an NFS share, having no problems. Have also ran several hobbit servers with the application binaries housed on NFS mounts; the app binaries & rrd files on NFS mounts; and just the rrd files on NFS mounts. Have been running Hobbit server with both the app binaries and rrd files on an NFS mount, for more than a year now. Unless there are some NFS performance issues on your network, housing those files on an NFS mount isn't the problem.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Sunday, June 24, 2007 3:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Got it working! I moved the data directory to the local disk and deleted the contents. After a while it all started working.
rrd obviously doesn't like nfs shares.
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Well, moving the log directory to the local disk stopped the error messages but I'm still getting empty graphs.
Dean, the requirements show PCRE, RRDtool, libpng, OpenSSL and OpenLDAP libraries. I've got all of those:
(from ./configure)
Checking for RRDtool ... test-rrd.c: In function 'main': test-rrd.c:30: error: too few arguments to function 'rrd_graph' make: *** [test-compile] Error 1 Not RRDtool 1.0.x, checking for 1.2.x Found RRDtool include files in /usr/include Found RRDtool libraries in /usr/lib Linking RRD with PNG library: -L/usr/lib -lpng
Checking for PCRE ... Found PCRE include files in /usr/include Found PCRE libraries in /usr/lib
Checking for OpenSSL ... Found OpenSSL include files in /usr/include Found OpenSSL libraries in /usr/lib
Any more ideas?
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote: > 2007-06-22 19:24:55 RRD error updating > /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: > could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Mon, Jun 25, 2007 at 08:34:13AM +0100, Robin Wood wrote:
Well, something fixed it and I'd rather not break again it while fiddling around to try to find out what it was!
You could try and run "rpcinfo -p" and see if there's an "nlockmgr" service registered - this is what normally handles NFS locking requests.
Regards, Henrik
Did you run "make clean" before recompiling? What OS are you running the hobbit server on?
Dean Casey
P A T H F I R E
Email: dcasey at pathfire.com
Office: 678 566 5664
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Sunday, June 24, 2007 8:23 AM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Well, moving the log directory to the local disk stopped the error messages but I'm still getting empty graphs.
Dean, the requirements show PCRE, RRDtool, libpng, OpenSSL and OpenLDAP libraries. I've got all of those:
(from ./configure)
Checking for RRDtool ... test-rrd.c: In function 'main': test-rrd.c:30: error: too few arguments to function 'rrd_graph' make: *** [test-compile] Error 1 Not RRDtool 1.0.x, checking for 1.2.x Found RRDtool include files in /usr/include Found RRDtool libraries in /usr/lib Linking RRD with PNG library: -L/usr/lib -lpng
Checking for PCRE ... Found PCRE include files in /usr/include Found PCRE libraries in /usr/lib
Checking for OpenSSL ... Found OpenSSL include files in /usr/include Found OpenSSL libraries in /usr/lib
Any more ideas?
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Did you run "make clean" before recompiling? What OS are you running the hobbit server on?
I didn't recompile, just moved the rrd data file off the nfs share to a local disk.
I'm running archlinux with a 2.6.19-beyond kernel
Robin
Dean Casey
P A T H F I R E
Email: dcasey at pathfire.com
Office: 678 566 5664
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Sunday, June 24, 2007 8:23 AM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Well, moving the log directory to the local disk stopped the error messages but I'm still getting empty graphs.
Dean, the requirements show PCRE, RRDtool, libpng, OpenSSL and OpenLDAP libraries. I've got all of those:
(from ./configure)
Checking for RRDtool ... test-rrd.c: In function 'main': test-rrd.c:30: error: too few arguments to function 'rrd_graph' make: *** [test-compile] Error 1 Not RRDtool 1.0.x, checking for 1.2.x Found RRDtool include files in /usr/include Found RRDtool libraries in /usr/lib Linking RRD with PNG library: -L/usr/lib -lpng
Checking for PCRE ... Found PCRE include files in /usr/include Found PCRE libraries in /usr/lib
Checking for OpenSSL ... Found OpenSSL include files in /usr/include Found OpenSSL libraries in /usr/lib
Any more ideas?
Robin
On 6/24/07, Robin Wood <dninja at gmail.com> wrote:
Hi What I should have added is that I am seeing graphs but with no graph lines and no numeric data in them. The filesystem graph for example, shows all my mount points but then where the numbers are supposed to be it is showing nan.
I'm trying moving the data/rrd directory to a local disk now, I'll let you know how it gets on.
Robin
On 6/24/07, Dean Casey <dcasey at pathfire.com> wrote:
Rrdtool by itself is not sufficient for all the libraries hobbit needs. Please check the list of installed libraries against the hobbit pre-requisites & install any missing libraries. Then, when you re-compile hobbit afterward, run "make clean" before running "make" and "make install". The "make clean" command is important.
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Saturday, June 23, 2007 12:53 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
I built hobbit from source and initially it complained that the libraries weren't there so I installed the rrdtool package and tried again. This time it built so I assume that it had all it needed.
If you can let me know which files need what permissions I'll check when I check the nfs share suggestion later.
Robin
On 6/23/07, Dean Casey <dcasey at pathfire.com> wrote:
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Indeed, we get this when something in our MRTG process gets an error, and the rrd data files then become owned by root instead of hobbit. We also get it if we start hobbit as root instead of hobbit. I think the lock is a red herring - it simply could not open the file.
My fix has been to do cd /var/lib/hobbit/rrd (or wherever you keep your rrd data) and then chown hobbit * -R
As far as the MRTG causing it, the MRTG logs generally point the way.
The key seems to be to configure and operate hobbit as the hobbit user.
=Seymour=
Seymour J. Brown Network Analyst CCNA, CFS, CWNA, Network+ West Jefferson Medical Center 1101 Medical Center Blvd. Marrero, LA 70072 504-349-2396
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Saturday, June 23, 2007 12:08 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. West Jefferson Medical Center, 1101 Medical Center Blvd., Marrero, USA, www.wjmc.org.
Seymour makes a good point. I also get similar errors when running Hobbit as root... doing that often changes file permissions and "breaks" Hobbit afterward.
The solution was to set up a very simple startup file that looks something like this (on RedHat, other Linuxes will vary but should be pretty close), based on the old BB startup script:
#!/bin/bash
hobbit-server This shell script takes care of starting and stopping
hobbit monitoring server.
chkconfig: - 99 45
description: Hobbit monitoring server is\
used to monitor Pathfire's systems.
probe: true
Source function library.
. /etc/rc.d/init.d/functions
Source networking configuration.
. /etc/sysconfig/network
Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
PATH=$PATH:/bin:/usr/bin # FOR su BBHOME=/home/hobbit BBUSER=hobbit
case "$1" in start) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh start" ;; restart) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh restart" ;; stop) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh stop" ;; status) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh status" ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 ;; esac
exit 0
I then set permissions on the startup file so: -rwxr-xr-x 1 root root 954 Jun 19 11:49 hobbit-server
I then ran: "chkconfig --345 hobbit-server on"
Now, on RedHat, I can run hobbit server without worrying about "su - hobbit" first, using commands: "service hobbit-server (start|stop|restart|status)"
You can do something similar for the hobbit client on RedHat or other Linux variants. Solaris would use a truncated version of this startup script.
Dean Casey
-----Original Message----- From: Brown, Seymour [mailto:Seymour.Brown at WJMC.ORG] Sent: Monday, June 25, 2007 10:19 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Indeed, we get this when something in our MRTG process gets an error, and the rrd data files then become owned by root instead of hobbit. We also get it if we start hobbit as root instead of hobbit. I think the lock is a red herring - it simply could not open the file.
My fix has been to do cd /var/lib/hobbit/rrd (or wherever you keep your rrd data) and then chown hobbit * -R
As far as the MRTG causing it, the MRTG logs generally point the way.
The key seems to be to configure and operate hobbit as the hobbit user.
=Seymour=
Seymour J. Brown Network Analyst CCNA, CFS, CWNA, Network+ West Jefferson Medical Center 1101 Medical Center Blvd. Marrero, LA 70072 504-349-2396
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Saturday, June 23, 2007 12:08 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. West Jefferson Medical Center, 1101 Medical Center Blvd., Marrero, USA, www.wjmc.org.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
I thought the hobbit startup script already checked for being run as root (I know if I try to run as root it tells me to run it as the hobbit user)...unless it was configured to do so at compile time. NFS generally can't handle being updated as root unless you explicitly configure the server to allow it in the mount config (this varies from system to system so check out the manpage). This would explain the behavior you're seeing when running hobbiut as root.
=G=
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Monday, June 25, 2007 10:44 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Seymour makes a good point. I also get similar errors when running Hobbit as root... doing that often changes file permissions and "breaks" Hobbit afterward.
The solution was to set up a very simple startup file that looks something like this (on RedHat, other Linuxes will vary but should be pretty close), based on the old BB startup script:
#!/bin/bash
hobbit-server This shell script takes care of starting and stopping
hobbit monitoring server.
chkconfig: - 99 45
description: Hobbit monitoring server is\ # used to monitor Pathfire's systems.
probe: true
Source function library.
. /etc/rc.d/init.d/functions
Source networking configuration.
. /etc/sysconfig/network
Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
PATH=$PATH:/bin:/usr/bin # FOR su BBHOME=/home/hobbit BBUSER=hobbit
case "$1" in start) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh start" ;; restart) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh restart" ;; stop) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh stop" ;; status) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh status" ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 ;; esac
exit 0
I then set permissions on the startup file so: -rwxr-xr-x 1 root root 954 Jun 19 11:49 hobbit-server
I then ran: "chkconfig --345 hobbit-server on"
Now, on RedHat, I can run hobbit server without worrying about "su - hobbit" first, using commands: "service hobbit-server (start|stop|restart|status)"
You can do something similar for the hobbit client on RedHat or other Linux variants. Solaris would use a truncated version of this startup script.
Dean Casey
-----Original Message----- From: Brown, Seymour [mailto:Seymour.Brown at WJMC.ORG] Sent: Monday, June 25, 2007 10:19 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Indeed, we get this when something in our MRTG process gets an error, and the rrd data files then become owned by root instead of hobbit. We also get it if we start hobbit as root instead of hobbit. I think the lock is a red herring - it simply could not open the file.
My fix has been to do cd /var/lib/hobbit/rrd (or wherever you keep your rrd data) and then chown hobbit * -R
As far as the MRTG causing it, the MRTG logs generally point the way.
The key seems to be to configure and operate hobbit as the hobbit user.
=Seymour=
Seymour J. Brown Network Analyst CCNA, CFS, CWNA, Network+ West Jefferson Medical Center 1101 Medical Center Blvd. Marrero, LA 70072 504-349-2396
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Saturday, June 23, 2007 12:08 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. West Jefferson Medical Center, 1101 Medical Center Blvd., Marrero, USA, www.wjmc.org.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Hobbit definitely wasn't being ran as root and the rrd directories were created either by hobbit or by the install script. The version I'm running at the moment is on a test system, when I roll it out to the new live box I'll see if it happens again.
Robin
On 6/25/07, Galen Johnson <Galen.Johnson at sas.com> wrote:
I thought the hobbit startup script already checked for being run as root (I know if I try to run as root it tells me to run it as the hobbit user)...unless it was configured to do so at compile time. NFS generally can't handle being updated as root unless you explicitly configure the server to allow it in the mount config (this varies from system to system so check out the manpage). This would explain the behavior you're seeing when running hobbiut as root.
=G=
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Monday, June 25, 2007 10:44 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Seymour makes a good point. I also get similar errors when running Hobbit as root... doing that often changes file permissions and "breaks" Hobbit afterward.
The solution was to set up a very simple startup file that looks something like this (on RedHat, other Linuxes will vary but should be pretty close), based on the old BB startup script:
#!/bin/bash
hobbit-server This shell script takes care of starting and stopping
hobbit monitoring server.
chkconfig: - 99 45
description: Hobbit monitoring server is\ # used to monitor Pathfire's systems.
probe: true
Source function library.
. /etc/rc.d/init.d/functions
Source networking configuration.
. /etc/sysconfig/network
Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
PATH=$PATH:/bin:/usr/bin # FOR su BBHOME=/home/hobbit BBUSER=hobbit
case "$1" in start) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh start" ;; restart) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh restart" ;; stop) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh stop" ;; status) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh status" ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 ;; esac
exit 0
I then set permissions on the startup file so: -rwxr-xr-x 1 root root 954 Jun 19 11:49 hobbit-server
I then ran: "chkconfig --345 hobbit-server on"
Now, on RedHat, I can run hobbit server without worrying about "su - hobbit" first, using commands: "service hobbit-server (start|stop|restart|status)"
You can do something similar for the hobbit client on RedHat or other Linux variants. Solaris would use a truncated version of this startup script.
Dean Casey
-----Original Message----- From: Brown, Seymour [mailto:Seymour.Brown at WJMC.ORG] Sent: Monday, June 25, 2007 10:19 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Indeed, we get this when something in our MRTG process gets an error, and the rrd data files then become owned by root instead of hobbit. We also get it if we start hobbit as root instead of hobbit. I think the lock is a red herring - it simply could not open the file.
My fix has been to do cd /var/lib/hobbit/rrd (or wherever you keep your rrd data) and then chown hobbit * -R
As far as the MRTG causing it, the MRTG logs generally point the way.
The key seems to be to configure and operate hobbit as the hobbit user.
=Seymour=
Seymour J. Brown Network Analyst CCNA, CFS, CWNA, Network+ West Jefferson Medical Center 1101 Medical Center Blvd. Marrero, LA 70072 504-349-2396
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Saturday, June 23, 2007 12:08 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. West Jefferson Medical Center, 1101 Medical Center Blvd., Marrero, USA, www.wjmc.org.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Possibly the NFS server was set to use LDAP or some other authentication method? Dose the NFS serer have a same uid/gid user account as the hobbit account on the monitoring server? Is the NFS set up to allow "777" and root-level permissions for the monitoring server (when accessing the NFS share)? This is starting to sound more and more like an NFS authentication conflict.
Dean
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Monday, June 25, 2007 12:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hobbit definitely wasn't being ran as root and the rrd directories were created either by hobbit or by the install script. The version I'm running at the moment is on a test system, when I roll it out to the new live box I'll see if it happens again.
Robin
On 6/25/07, Galen Johnson <Galen.Johnson at sas.com> wrote:
I thought the hobbit startup script already checked for being run as root (I know if I try to run as root it tells me to run it as the hobbit user)...unless it was configured to do so at compile time. NFS generally can't handle being updated as root unless you explicitly configure the server to allow it in the mount config (this varies from system to system so check out the manpage). This would explain the behavior you're seeing when running hobbiut as root.
=G=
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Monday, June 25, 2007 10:44 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Seymour makes a good point. I also get similar errors when running Hobbit as root... doing that often changes file permissions and "breaks" Hobbit afterward.
The solution was to set up a very simple startup file that looks something like this (on RedHat, other Linuxes will vary but should be pretty close), based on the old BB startup script:
#!/bin/bash
hobbit-server This shell script takes care of starting and
stopping
hobbit monitoring server.
chkconfig: - 99 45
description: Hobbit monitoring server is\ # used to monitor
Pathfire's systems.
probe: true
Source function library.
. /etc/rc.d/init.d/functions
Source networking configuration.
. /etc/sysconfig/network
Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
PATH=$PATH:/bin:/usr/bin # FOR su BBHOME=/home/hobbit BBUSER=hobbit
case "$1" in start) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh start" ;; restart) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh restart" ;; stop) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh stop" ;; status) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh status" ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 ;; esac
exit 0
I then set permissions on the startup file so: -rwxr-xr-x 1 root root 954 Jun 19 11:49 hobbit-server
I then ran: "chkconfig --345 hobbit-server on"
Now, on RedHat, I can run hobbit server without worrying about "su - hobbit" first, using commands: "service hobbit-server (start|stop|restart|status)"
You can do something similar for the hobbit client on RedHat or other Linux variants. Solaris would use a truncated version of this startup script.
Dean Casey
-----Original Message----- From: Brown, Seymour [mailto:Seymour.Brown at WJMC.ORG] Sent: Monday, June 25, 2007 10:19 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Indeed, we get this when something in our MRTG process gets an error, and the rrd data files then become owned by root instead of hobbit. We also get it if we start hobbit as root instead of hobbit. I think the lock is a red herring - it simply could not open the file.
My fix has been to do cd /var/lib/hobbit/rrd (or wherever you keep your rrd data) and then chown hobbit * -R
As far as the MRTG causing it, the MRTG logs generally point the way.
The key seems to be to configure and operate hobbit as the hobbit user.
=Seymour=
Seymour J. Brown Network Analyst CCNA, CFS, CWNA, Network+ West Jefferson Medical Center 1101 Medical Center Blvd. Marrero, LA 70072 504-349-2396
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Saturday, June 23, 2007 12:08 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. West Jefferson Medical Center, 1101 Medical Center Blvd., Marrero, USA, www.wjmc.org.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
NFS is using normal user authentication, nothing special. Both machines have hobbit as a user with the same uid.
I've just put it back onto the nfs share and it is complaining again.
The rrd directory is there and set at 777. Something, I assume rrd, created all the files in the directory so it has write access. After it created them all I've set them all to 777 so anyone can do anything with them, still getting errors.
ls -alR .: total 1 drwxrwxrwx 5 hobbit users 120 2007-06-26 09:59 . drwxr-xr-x 10 hobbit users 240 2007-06-26 09:50 .. drwxrwxrwx 2 hobbit users 536 2007-06-26 10:00 ra
./ra: total 701 drwxrwxrwx 2 hobbit users 536 2007-06-26 10:00 . drwxrwxrwx 5 hobbit users 120 2007-06-26 09:59 .. -rwxrwxrwx 1 hobbit users 19552 2007-06-26 09:58 bbgen.rrd -rwxrwxrwx 1 hobbit users 19552 2007-06-26 09:57 clock.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:57 disk,root.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:58 disk,var,cache,pacman.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:58 disk,var,iso.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:59 disk,var,vmware.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:57 ifstat.eth0.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:57 ifstat.vmnet1.rrd -rwxrwxrwx 1 hobbit users 38536 2007-06-26 09:58 ifstat.vmnet8.rrd -rwxrwxrwx 1 hobbit users 304312 2007-06-26 09:56 netstat.rrd -rwxrwxrwx 1 hobbit users 19552 2007-06-26 09:56 procs.rrd -rw-r--r-- 1 hobbit users 19552 2007-06-26 10:00 tcp.bbd.rrd -rwxrwxrwx 1 hobbit users 19552 2007-06-26 10:00 tcp.conn.rrd -rwxrwxrwx 1 hobbit users 19552 2007-06-26 09:56 users.rrd
An example error message:
2007-06-26 10:00:31 RRD error updating /var/sites/hobbit/non-secure//data/rrd/rastlin/tcp.conn.rrd from 192.168.0.2: could not lock RRD
Where would the lock file be created? Is that in a different directory?
Robin
On 6/25/07, Dean Casey <dcasey at pathfire.com> wrote:
Possibly the NFS server was set to use LDAP or some other authentication method? Dose the NFS serer have a same uid/gid user account as the hobbit account on the monitoring server? Is the NFS set up to allow "777" and root-level permissions for the monitoring server (when accessing the NFS share)? This is starting to sound more and more like an NFS authentication conflict.
Dean
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Monday, June 25, 2007 12:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hobbit definitely wasn't being ran as root and the rrd directories were created either by hobbit or by the install script. The version I'm running at the moment is on a test system, when I roll it out to the new live box I'll see if it happens again.
Robin
On 6/25/07, Galen Johnson <Galen.Johnson at sas.com> wrote:
I thought the hobbit startup script already checked for being run as root (I know if I try to run as root it tells me to run it as the hobbit user)...unless it was configured to do so at compile time. NFS generally can't handle being updated as root unless you explicitly configure the server to allow it in the mount config (this varies from system to system so check out the manpage). This would explain the behavior you're seeing when running hobbiut as root.
=G=
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Monday, June 25, 2007 10:44 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Seymour makes a good point. I also get similar errors when running Hobbit as root... doing that often changes file permissions and "breaks" Hobbit afterward.
The solution was to set up a very simple startup file that looks something like this (on RedHat, other Linuxes will vary but should be pretty close), based on the old BB startup script:
#!/bin/bash
hobbit-server This shell script takes care of starting and
stopping
hobbit monitoring server.
chkconfig: - 99 45
description: Hobbit monitoring server is\ # used to monitor
Pathfire's systems.
probe: true
Source function library.
. /etc/rc.d/init.d/functions
Source networking configuration.
. /etc/sysconfig/network
Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
PATH=$PATH:/bin:/usr/bin # FOR su BBHOME=/home/hobbit BBUSER=hobbit
case "$1" in start) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh start" ;; restart) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh restart" ;; stop) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh stop" ;; status) su - $BBUSER -c "cd $BBHOME;./server/hobbit.sh status" ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 ;; esac
exit 0
I then set permissions on the startup file so: -rwxr-xr-x 1 root root 954 Jun 19 11:49 hobbit-server
I then ran: "chkconfig --345 hobbit-server on"
Now, on RedHat, I can run hobbit server without worrying about "su - hobbit" first, using commands: "service hobbit-server (start|stop|restart|status)"
You can do something similar for the hobbit client on RedHat or other Linux variants. Solaris would use a truncated version of this startup script.
Dean Casey
-----Original Message----- From: Brown, Seymour [mailto:Seymour.Brown at WJMC.ORG] Sent: Monday, June 25, 2007 10:19 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
Indeed, we get this when something in our MRTG process gets an error, and the rrd data files then become owned by root instead of hobbit. We also get it if we start hobbit as root instead of hobbit. I think the lock is a red herring - it simply could not open the file.
My fix has been to do cd /var/lib/hobbit/rrd (or wherever you keep your rrd data) and then chown hobbit * -R
As far as the MRTG causing it, the MRTG logs generally point the way.
The key seems to be to configure and operate hobbit as the hobbit user.
=Seymour=
Seymour J. Brown Network Analyst CCNA, CFS, CWNA, Network+ West Jefferson Medical Center 1101 Medical Center Blvd. Marrero, LA 70072 504-349-2396
-----Original Message----- From: Dean Casey [mailto:dcasey at pathfire.com] Sent: Saturday, June 23, 2007 12:08 AM To: hobbit at hswn.dk Subject: RE: [hobbit] I'm getting the error "could not lock RRD"
I've seen this rrd error in the past when rrd libraries, binaries, or data files had incorrect permissions or were missing (unavailable).
Dean Casey
-----Original Message----- From: Robin Wood [mailto:dninja at gmail.com] Sent: Friday, June 22, 2007 4:59 PM To: hobbit at hswn.dk Subject: Re: [hobbit] I'm getting the error "could not lock RRD"
Hi The directory is a nfs share. I'll try moving the data directory to a local disk and see if that fixes it.
Robin
On 6/22/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Jun 22, 2007 at 07:27:27PM +0100, Robin Wood wrote:
2007-06-22 19:24:55 RRD error updating /var/sites/hobbit/non-secure//data/rrd/xxx/procs.rrd from 192.168.0.2: could not lock RRD
It's an error from the rrdtool library. I honestly have no idea what could be causing this.
Is the /var/sites/hobbit/non-secure//data/rrd an NFS filesystem? Perhaps there's some issue with locking on networked filsystems.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. West Jefferson Medical Center, 1101 Medical Center Blvd., Marrero, USA, www.wjmc.org.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
participants (5)
-
dcasey＠pathfire.com
-
dninja＠gmail.com
-
Galen.Johnson＠sas.com
-
henrik＠hswn.dk
-
Seymour.Brown＠WJMC.ORG