mounted NFS volumes not included in disk utilization graphs
Den 31-03-2015 kl. 00:49 skrev Paul Grondahl:
On several systems with NFS mounts, the mounted volumes are not included in the disk utilization graphs. The volumes are visible in the client data. How does one get them included in the graphs? There is a potential problem with including network disks in the "disk" status: If your network connection to the NFS server dies, then the "df" command is likely to stall. So you won't get any client data sent over to the Xymon server, and your entire server goes purple.
That is why the Xymon client uses the "-l" (local) flag for df to NOT report on network filesystems.
Regards, Henrik
I'll "me, too" Henrik's response (good to see he's still lurking). df on nfs should happen on the server side. Another problem, I've had the same thing happen to SAN disks that were "locally" attached when the server drops away unexpectedly.
=G=
From: Xymon <xymon-bounces at xymon.com> on behalf of Henrik Størner <henrik at hswn.dk> Sent: Wednesday, April 1, 2015 4:13 PM To: xymon at xymon.com Subject: Re: [Xymon] mounted NFS volumes not included in disk utilization graphs
Den 31-03-2015 kl. 00:49 skrev Paul Grondahl:
On several systems with NFS mounts, the mounted volumes are not included in the disk utilization graphs. The volumes are visible in the client data. How does one get them included in the graphs? There is a potential problem with including network disks in the "disk" status: If your network connection to the NFS server dies, then the "df" command is likely to stall. So you won't get any client data sent over to the Xymon server, and your entire server goes purple.
That is why the Xymon client uses the "-l" (local) flag for df to NOT report on network filesystems.
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
At my work, my team developed some "safe" coding to list NFS filesystems in df. It uses the showmount command to list exports on the NFS server - if this succeeds, it should be safe to run df on the filesystem.
The comment in the code references the URL http://osdir.com/ml/monitoring.hobbit/2007/msg02373.html but it doesn't seem to exist now.
It's ugly and no doubt could be cleaned up and improved... but it does work, and it's handy to know which clients are affected when a NFS volume becomes full.
Update to xymonclient-linux.sh :
At the end of the code for "df", add these lines :
--- Start - Addition code, to check nfs mounts ---
Added following code, based on http://osdir.com/ml/monitoring.hobbit/2007/msg02373.html
/bin/bash -c "/bin/egrep $'[ \t]nfs[ \t]' /etc/mtab" |
/bin/egrep -v "ignore|indirect" |
/usr/bin/awk '{print $1, $2}' |
while read line
do
set -- $line
NFSHOST=echo $1 | /usr/bin/awk -F: '{print $1}'
SHARE=echo $1 | /usr/bin/awk -F: '{print $2}'
/bin/ping $NFSHOST -c 5 >/dev/null 2>&1
if [ $? -ne 0 ]
then
echo "$1 IS NOT RESPONDING(ping_failed) 999% $2"
else
# Ensure NFS connection is active BEFORE running the "showmount -e" command.
/bin/df -k $2 > /dev/null 2>&1 & /bin/sleep 5 ; kill $! > /dev/null 2>&1
/usr/sbin/showmount -e $NFSHOST >/dev/null 2>&1
if [ $? -ne 0 ]; then
echo "$1 IS NOT AVAILABLE(showmount_failed) 999% $2"
else
/usr/sbin/showmount -e $NFSHOST | /bin/egrep "^$SHARE " >/dev/null 2>&1
if [ $? -ne 0 ]; then
# Remove "/share" prefix as required for QNAP NFS servers
SHARE1=${SHARE#/share}
/usr/sbin/showmount -e $NFSHOST | /bin/egrep "^$SHARE1 " >/dev/null 2>&1
fi
if [ $? -ne 0 ]; then
echo "$1 IS NOT AVAILABLE(showmount_failed_to_find_share) 999% $2"
else
/bin/df -kP $2 | tail -n +2
fi
fi
fi
done
--- End of - Addition code, to check nfs mounts ---
Regards Mal
On 02/04/15 07:13, Henrik Størner wrote:
Den 31-03-2015 kl. 00:49 skrev Paul Grondahl:
On several systems with NFS mounts, the mounted volumes are not included in the disk utilization graphs. The volumes are visible in the client data. How does one get them included in the graphs? There is a potential problem with including network disks in the "disk" status: If your network connection to the NFS server dies, then the "df" command is likely to stall. So you won't get any client data sent over to the Xymon server, and your entire server goes purple.
That is why the Xymon client uses the "-l" (local) flag for df to NOT report on network filesystems.
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Thanks all for your detailed responses. I will just monitor the NFS host.
On Apr 1, 2015, at 6:41 PM, Malcolm <fragbaitmm at gmail.com> wrote:
At my work, my team developed some "safe" coding to list NFS filesystems in df. It uses the showmount command to list exports on the NFS server - if this succeeds, it should be safe to run df on the filesystem.
The comment in the code references the URL http://osdir.com/ml/monitoring.hobbit/2007/msg02373.html but it doesn't seem to exist now.
It's ugly and no doubt could be cleaned up and improved... but it does work, and it's handy to know which clients are affected when a NFS volume becomes full.
Update to xymonclient-linux.sh :
At the end of the code for "df", add these lines :
--- Start - Addition code, to check nfs mounts ---
Added following code, based on http://osdir.com/ml/monitoring.hobbit/2007/msg02373.html
/bin/bash -c "/bin/egrep $'[ \t]nfs[ \t]' /etc/mtab" |
/bin/egrep -v "ignore|indirect" |
/usr/bin/awk '{print $1, $2}' |
while read line do set -- $line NFSHOST=echo $1 | /usr/bin/awk -F: '{print $1}'SHARE=echo $1 | /usr/bin/awk -F: '{print $2}'/bin/ping $NFSHOST -c 5 >/dev/null 2>&1 if [ $? -ne 0 ] then echo "$1 IS NOT RESPONDING(ping_failed) 999% $2" else # Ensure NFS connection is active BEFORE running the "showmount -e" command. /bin/df -k $2 > /dev/null 2>&1 & /bin/sleep 5 ; kill $! > /dev/null 2>&1 /usr/sbin/showmount -e $NFSHOST >/dev/null 2>&1 if [ $? -ne 0 ]; then echo "$1 IS NOT AVAILABLE(showmount_failed) 999% $2" else /usr/sbin/showmount -e $NFSHOST | /bin/egrep "^$SHARE " >/dev/null 2>&1 if [ $? -ne 0 ]; then # Remove "/share" prefix as required for QNAP NFS servers SHARE1=${SHARE#/share} /usr/sbin/showmount -e $NFSHOST | /bin/egrep "^$SHARE1 " >/dev/null 2>&1 fi if [ $? -ne 0 ]; then echo "$1 IS NOT AVAILABLE(showmount_failed_to_find_share) 999% $2" else /bin/df -kP $2 | tail -n +2 fi fi fi done--- End of - Addition code, to check nfs mounts ---
Regards Mal
On 02/04/15 07:13, Henrik Størner wrote:
Den 31-03-2015 kl. 00:49 skrev Paul Grondahl:
On several systems with NFS mounts, the mounted volumes are not included in the disk utilization graphs. The volumes are visible in the client data. How does one get them included in the graphs? There is a potential problem with including network disks in the "disk" status: If your network connection to the NFS server dies, then the "df" command is likely to stall. So you won't get any client data sent over to the Xymon server, and your entire server goes purple.
That is why the Xymon client uses the "-l" (local) flag for df to NOT report on network filesystems.
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
participants (4)
-
fragbaitmm＠gmail.com
-
Galen.Johnson＠sas.com
-
henrik＠hswn.dk
-
paul＠arrowtel.net