Hi all
Has anybody done any work on monitoring an EMC Clariion system, that they are prepared to share? I have no idea what I need to monitor yet, and we have nothing in place at the moment, so anything is better than what I have now.
All assistance appreciated.
Regards Vernon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
There's this rather crude script I learned much more perl since I wrote it and looking at it now there's a bunch of stuff I want to fix, but you can play with it (I am going to clean it up, and add custom per-device storage levels, ask me in a while if I've done that :-)
Assumes youv'e set up sshkey access for xymon user -> "nasadmin" on your devices
#!/usr/bin/perl
Script to test EMC storage devices in xymon
2011/11/08 1.0 - betsys - initial install
use strict; use warnings; use DBI; use Time::Local;
#use Net::SSH::Perl;
use constant false => 0; use constant true => 1;
my $REDLEVEL = 95; my $YELLOWLEVEL = 90;
my $REDDOT = '<IMG SRC="/xymon/gifs/red-recent.gif">'; my $YELLOWDOT = '<IMG SRC="/xymon/gifs/yellow-recent.gif">';
#-------- Configure my $BBHOME = ENV{BBHOME}; # Path to the BBServer software. my $BBTMP = "$BBHOME/tmp"; my $BB = $ENV{'BB'}; my $BBDISP = $ENV{'BBDISP'}; my $BBDISPLAYS = $ENV{'BBDISPLAYS'}; my $TESTNAME = "temp"; # Test name as displayed on the main page (bb.html).
#-------- End Configure
my $COLOR = "clear"; my $BBTMPLOG = "$BBHOME/tmp/TESTNAME.tmp"; my $BBSTATLOG = "$BBHOME/tmp/TESTNAME"; my $epoch = time(); my $date = localtime($epoch); my $status = "clear"; my $gored = false; my $goyellow = false; my $nasadmin = "nasadmin";
my $server = ""; # we're using strict my @serverlist = ( "server1.example.com", "server2.example.com", "server3.example.com" );
foreach $server (@serverlist) { test_uptime($server); test_storage($server); test_replication($server);
}
sub test_replication { my ($serv) = @_; $COLOR = "green"; $gored = false; $goyellow = false; my @warnings = ("\n");
my $TESTNAME = "nas_rep"; # Test name as displayed on the main
page (bb.html). my $nascmd = "export NAS_DB=/nas;/nas/bin/nas_replicate -list"; my $sshcmd = "ssh $nasadmin\@$serv \"$nascmd\"";
my @output = `$sshcmd >/tmp/$serv.nas_replicate.out `;
my @dataline = `cat /tmp/$serv.nas_replicate.out`;
open( FH, "< /tmp/$serv.nas_replicate.out" );
while (<FH>)
{
my @line = (split);
next if ( @line == 0 ); # skip blanks
next unless ( $line[1] =~ /filesystem/ );
if ( $line[5] ne "OK" )
{
$goyellow = true;
push( @warnings, "Problem with replication @line \n" );
}
}
if ( $gored == true ) { $COLOR = "red"; }
elsif ( $goyellow == true ) { $COLOR = "yellow"; }
else
{
$COLOR = "green";
push( @warnings, "Replication looks OK on $serv\n\n" );
}
my $bbcmd = "$BB $BBDISPLAY 'status+2h $serv.$TESTNAME $COLOR
$date \n @warnings \n @dataline' ";
# print "command is";
# print $bbcmd;
# print "\n";
system("$bbcmd");
}
sub test_uptime { my ($serv) = @_; $COLOR = "green"; my $TESTNAME = "uptime"; # Test name as displayed on the main page (bb.html). my $nascmd = "export NAS_DB=/nas;/nas/bin/server_uptime ALL"; my $sshcmd = "ssh nasadmin\@$serv \"$nascmd\"";
# my @output = `$sshcmd >/tmp/$serv.uptime.out 2 >& /dev/null`;
my @output = `$sshcmd >/tmp/$serv.uptime.out `;
# print "$server uptime output is @output \n";
open( FH, "< /tmp/$serv.uptime.out" );
while (<FH>)
{
$COLOR = "red" if /fault/;
}
my @dataline = `cat /tmp/$serv.uptime.out`;
my $bbcmd = "$BB $BBDISPLAY 'status+2h $serv.$TESTNAME $COLOR
$date \n @dataline' ";
# print "command is";
# print $bbcmd;
# print "\n";
system("$bbcmd");
}
sub test_storage { my ($serv) = @_; $COLOR = "green"; $gored = false; $goyellow = false; my @warnings = ("\n"); my $TESTNAME = "storage"; # Test name as displayed on the main page (bb.html). my $scratch = ""; my ( $fs, $kb, $used, $avail, $cap, $mount );
my $nascmd =
"export NAS_DB=/nas;/nas/bin/server_df ALL |grep -v iscsi| grep
-v root_ | grep -v ckpt | grep -v automaticND";
my $sshcmd = "ssh nasadmin\@$serv \"$nascmd\"";
my @output = $sshcmd >/tmp/$serv.storage.out ;
my @dataline = cat /tmp/$serv.storage.out;
# print "$serv storage output is @dataline \n";
open( FH, "< /tmp/$serv.storage.out" );
while (<FH>)
{
next if /server_/;
next if /Filesystem/;
my @line = (split);
if ( @line == 0 ) { next } # skip blanks
if ( @line == 1 )
{ # we might be on the first line
of a wrapped line $scratch = $line[0]; # but some one-word lines are ckpts followed by real lines next; } elsif ( ( @line >= 5 ) && ( $line[4] =~ /\%/ ) ) { # we have a filesystem data line ( $fs, $kb, $used, $avail, $cap, $mount ) = @line; } elsif ( ( $scratch ne '' ) && ( $line[3] =~ /\%/ ) ) { # we have the second line of a wrap $fs = $scratch; ( $kb, $used, $avail, $cap, $mount ) = @line; } else { $scratch = ""; next; } #at the moment, not interested in non-data lines
my $full = $cap; # number with pesky "%" appended
$full =~ s/%//; # is there a more elegant way to do this?
# print "used space is $full % \n";
if ( $full >= $REDLEVEL )
{
$gored = true;
push( @warnings,
"$REDDOT filesystem $fs ($cap used) has reached the
PANIC level ($REDLEVEL%) on mountpoint $mount \n" ); } elsif ( $full >= $YELLOWLEVEL ) { $goyellow = true; push( @warnings, "$YELLOWDOT filesystem $fs ($cap used) has reached the WARNING level ($YELLOWLEVEL%) on mountpoint $mount \n" ); }
# my $i=0;
# for ( $i=0; $i<@line; $i++){
# print "element #$i is $line[$i]\n";
# }
}
#!# print "$serv output is @output";
if ( $gored == true ) { $COLOR = "red"; }
elsif ( $goyellow == true ) { $COLOR = "yellow"; }
else { $COLOR = "green"; }
my $bbcmd = "$BB $BBDISPLAY 'status+2h $serv.$TESTNAME $COLOR
$date \n @warnings \n @dataline' ";
# print "command is";
# print $bbcmd;
# print "\n";
system("$bbcmd");
}
Hi Betsy
Thanks very much for that. Looking at it, I think you knew a lot more perl then than I know now :-) Even if it was the worst script you ever wrote, it's better than what I have now, which is nothing.
Thanks again.
Cheers Vernon
On 20 September 2012 23:48, Betsy Schwartz <betsy.schwartz at gmail.com> wrote:
There's this rather crude script I learned much more perl since I wrote it and looking at it now there's a bunch of stuff I want to fix, but you can play with it (I am going to clean it up, and add custom per-device storage levels, ask me in a while if I've done that :-)
Assumes youv'e set up sshkey access for xymon user -> "nasadmin" on your devices
#!/usr/bin/perl
Script to test EMC storage devices in xymon
2011/11/08 1.0 - betsys - initial install
use strict; use warnings; use DBI; use Time::Local;
#use Net::SSH::Perl;
use constant false => 0; use constant true => 1;
my $REDLEVEL = 95; my $YELLOWLEVEL = 90;
my $REDDOT = '<IMG SRC="/xymon/gifs/red-recent.gif">'; my $YELLOWDOT = '<IMG SRC="/xymon/gifs/yellow-recent.gif">';
#-------- Configure my $BBHOME = ENV{BBHOME}; # Path to the BBServer software. my $BBTMP = "$BBHOME/tmp"; my $BB = $ENV{'BB'}; my $BBDISP = $ENV{'BBDISP'}; my $BBDISPLAYS = $ENV{'BBDISPLAYS'}; my $TESTNAME = "temp"; # Test name as displayed on the main page (bb.html).
#-------- End Configure
my $COLOR = "clear"; my $BBTMPLOG = "$BBHOME/tmp/TESTNAME.tmp"; my $BBSTATLOG = "$BBHOME/tmp/TESTNAME"; my $epoch = time(); my $date = localtime($epoch); my $status = "clear"; my $gored = false; my $goyellow = false; my $nasadmin = "nasadmin";
my $server = ""; # we're using strict my @serverlist = ( "server1.example.com", "server2.example.com", "server3.example.com" );
foreach $server (@serverlist) { test_uptime($server); test_storage($server); test_replication($server);
}
sub test_replication { my ($serv) = @_; $COLOR = "green"; $gored = false; $goyellow = false; my @warnings = ("\n");
my $TESTNAME = "nas_rep"; # Test name as displayed on the mainpage (bb.html). my $nascmd = "export NAS_DB=/nas;/nas/bin/nas_replicate -list"; my $sshcmd = "ssh $nasadmin\@$serv \"$nascmd\"";
my @output = `$sshcmd >/tmp/$serv.nas_replicate.out `; my @dataline = `cat /tmp/$serv.nas_replicate.out`; open( FH, "< /tmp/$serv.nas_replicate.out" ); while (<FH>) { my @line = (split); next if ( @line == 0 ); # skip blanks next unless ( $line[1] =~ /filesystem/ ); if ( $line[5] ne "OK" ) { $goyellow = true; push( @warnings, "Problem with replication @line \n" ); } } if ( $gored == true ) { $COLOR = "red"; } elsif ( $goyellow == true ) { $COLOR = "yellow"; } else { $COLOR = "green"; push( @warnings, "Replication looks OK on $serv\n\n" ); } my $bbcmd = "$BB $BBDISPLAY 'status+2h $serv.$TESTNAME $COLOR$date \n @warnings \n @dataline' ";
# print "command is"; # print $bbcmd; # print "\n"; system("$bbcmd");}
sub test_uptime { my ($serv) = @_; $COLOR = "green"; my $TESTNAME = "uptime"; # Test name as displayed on the main page (bb.html). my $nascmd = "export NAS_DB=/nas;/nas/bin/server_uptime ALL"; my $sshcmd = "ssh nasadmin\@$serv \"$nascmd\"";
# my @output = `$sshcmd >/tmp/$serv.uptime.out 2 >& /dev/null`; my @output = `$sshcmd >/tmp/$serv.uptime.out `; # print "$server uptime output is @output \n"; open( FH, "< /tmp/$serv.uptime.out" ); while (<FH>) { $COLOR = "red" if /fault/; } my @dataline = `cat /tmp/$serv.uptime.out`; my $bbcmd = "$BB $BBDISPLAY 'status+2h $serv.$TESTNAME $COLOR$date \n @dataline' ";
# print "command is"; # print $bbcmd; # print "\n"; system("$bbcmd");}
sub test_storage { my ($serv) = @_; $COLOR = "green"; $gored = false; $goyellow = false; my @warnings = ("\n"); my $TESTNAME = "storage"; # Test name as displayed on the main page (bb.html). my $scratch = ""; my ( $fs, $kb, $used, $avail, $cap, $mount );
my $nascmd = "export NAS_DB=/nas;/nas/bin/server_df ALL |grep -v iscsi| grep-v root_ | grep -v ckpt | grep -v automaticND"; my $sshcmd = "ssh nasadmin\@$serv \"$nascmd\""; my @output =
$sshcmd >/tmp/$serv.storage.out; my @dataline =cat /tmp/$serv.storage.out;# print "$serv storage output is @dataline \n"; open( FH, "< /tmp/$serv.storage.out" ); while (<FH>) { next if /server_/; next if /Filesystem/; my @line = (split); if ( @line == 0 ) { next } # skip blanks if ( @line == 1 ) { # we might be on the first lineof a wrapped line $scratch = $line[0]; # but some one-word lines are ckpts followed by real lines next; } elsif ( ( @line >= 5 ) && ( $line[4] =~ /\%/ ) ) { # we have a filesystem data line ( $fs, $kb, $used, $avail, $cap, $mount ) = @line; } elsif ( ( $scratch ne '' ) && ( $line[3] =~ /\%/ ) ) { # we have the second line of a wrap $fs = $scratch; ( $kb, $used, $avail, $cap, $mount ) = @line; } else { $scratch = ""; next; } #at the moment, not interested in non-data lines
my $full = $cap; # number with pesky "%" appended $full =~ s/%//; # is there a more elegant way to do this? # print "used space is $full % \n"; if ( $full >= $REDLEVEL ) { $gored = true; push( @warnings, "$REDDOT filesystem $fs ($cap used) has reached thePANIC level ($REDLEVEL%) on mountpoint $mount \n" ); } elsif ( $full >= $YELLOWLEVEL ) { $goyellow = true; push( @warnings, "$YELLOWDOT filesystem $fs ($cap used) has reached the WARNING level ($YELLOWLEVEL%) on mountpoint $mount \n" ); }
# my $i=0; # for ( $i=0; $i<@line; $i++){ # print "element #$i is $line[$i]\n"; # } } #!# print "$serv output is @output"; if ( $gored == true ) { $COLOR = "red"; } elsif ( $goyellow == true ) { $COLOR = "yellow"; } else { $COLOR = "green"; } my $bbcmd = "$BB $BBDISPLAY 'status+2h $serv.$TESTNAME $COLOR$date \n @warnings \n @dataline' ";
# print "command is"; # print $bbcmd; # print "\n"; system("$bbcmd");}
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
Here's a script we used to monitor clariion. It requires the navicli to be installed on the host that runs the monitoring script. In our case it was the xymon server. We also found it better to split the test accross both service processors. Hence the SPIP and SPIP2 variables. Ours had no NAS so we didn't montitor anything there. This was on the old deadcat site and we tweaked it a little.
#!/bin/sh #set -x #-------------------------------------------------
Big Brother - EMC CLARiiON FC-4700 Hardware Agent
This agent monitor the hardware of an EMC
CLARiiON FC-4700 Array using the Navisphere
Command Line Interface (NAVICLI)
Author: Sabey (sabey2000 at hotmail.com)
Feel free to improve or modify.
#==================================================
History
#--------------------------------------------------
Fri Feb 22 11:44:08 MET 2008
added global state yellow and Disk status
Notes:
If a disk failed, it was shown yellow in it`s list, but the global
status was unchanged. This is fixed now
The Disks will show now the status of a binding action in %
the same for replacement of a disk
Tue Aug 19 21:34:15 GMT 2003
Docs and fixes by N. Rettinghouse (rettinghouse.n0spam at att.net)
Notes:
GNU GAWK is needed. (www.sunfreeware.com for Solaris)
Copy clariion.sh into the bb/ext directory.
Add the following to your bb/etc/bb-bbexttab:
localhost : : clariion.sh
A restart of BigBro will be needed after adding bb-bbexttab.
Configure the variables shown below.
Add the BigBrother user to the /etc/Navisphere/agent.config file to
allow running of navicli.
Changed "HW" to "hw" to be consistant with other bb services.
Fixed conflict with $MACHINE BigBro internal variable.
Configure bb-hosts like this:
10.10.30.34 SAN-BigusDiskus # testip noping
#---------------------------------------------------
Thu Sep 4 11:57:48 EDT 2003
Docs and fixes by Sabey (sabey2000 at hotmail.com)
Notes:
Add support to CLARiiON CX600 (maybe all CX series...)
#############################################################################
Set these as needed.
#S#SPIP=172.25.2.239 # One of the two SP IP address #S#EMC_ARRAY="CLARiiON-1" # The name of your CLARiiON array SPIP= # One of the two SP IP address SPIP2= # One of the two SP IP address EMC_ARRAY="emcsan" # The name of your CLARiiON array navicli=/opt/Navisphere/bin/navicli # Where is the NAVICLI cmd GAWK=/usr/bin/awk # need GNU AWK ############################################################################# #############################################################################
TEST="emc" # The BB test name MYTMPFILE=CLARIION_STATUS.$$ # Use a buffer tmp file for NAVICLI output BBPROG=BB_CLARiiON; export BBPROG # ???
##---[Check BBHOME is set]-------------------------------- if test "$BBHOME" = ""; then echo "BBHOME is not set... exiting"; exit 1 fi if [ ! -x $GAWK ]; then echo "This agent need GNU Awk, check GAWK variable"; exit 1 fi
#---[Load bb global parameter]--------------------------- if test ! "$BBTMP"; then . $BBHOME/etc/bbdef.sh fi
#---[Collect DATA]--------------------------------------- echo "Status of ${EMC_ARRAY} Storage System" > $BBTMP/ITC_CLARiiON.OUT.$$ echo "============================================================================="
$BBTMP/ITC_CLARiiON.OUT.$$
#---[1. Get cache status]--------------------------------------------------------- $navicli -h $SPIP getcache > $BBTMP/$MYTMPFILE echo "SP Cache" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /SP Write|Read Cache State/) { sub (" *"," ") if ($0 ~ /State Enabled/) { sub (" State Enabled","") sub ("^","\\&green ") } else { sub (" State .*","") sub ("^","\\&red ") } if ($0 !~ /SP Read Cache/) { print } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[2. Port status]-------------------------------------------------------------- $navicli -h $SPIP2 port -list > $BBTMP/$MYTMPFILE echo -e "\nSP Link Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /Information about each SPPORT:/) { FLAG=1 } if (FLAG == 1) { if (($0 ~ /SP Name/) || ($0 ~ /SP Port ID/) || ($0 ~ /Link Status/) || ($0 ~ /Port Status/)) { sub (" *","") if ($0 ~ /SP Name/) { sub ("SP Name:","") SP=$0 } if ($0 ~ /SP Port ID/) { sub ("SP Port ID:","Port ") Port=$0 } if ($0 ~ /Link Status/) { if ($0 ~ /Status:Up/) { sub ("Status:Up","\\&green") } else { if ((Port == "Port 3") || (Port == "Port 2" )) { sub ("Status:.*","\\&clear") } else { sub ("Status:.*","\\&red") } } LinkS=$0 } if ($0 ~ /Port Status/) { if ($0 ~ /Status:Online/) { sub ("Status:Online","\\&green") } else { if ((Port == "Port 3") || (Port == "Port 2" )) { sub ("Status:.*","\\&clear") } else { sub ("Status:.*","\\&red") } } sub ("Port","Status") PortS=$0 sub ("Status:","",PortS) sub ("Status:","",LinkS) print SP,Port,PortS,LinkS } } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[3. DPE/DAE Status]----------------------------------------------------------- $navicli -h $SPIP getcrus > $BBTMP/$MYTMPFILE echo -e "\nDPE/DAE Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$
$CAT $BBTMP/$MYTMPFILE | $GAWK ' { if (($0 ~ /[DPE|DAE] Bus . Enclosure ./) || ($0 ~ /State/)) { if ($0 ~ /State/) { sub ("Bus . Enclosure . ","") sub (" *"," ") sub ("Power","Power Supply") if (($0 ~ /State: Present/) || ($0 ~ /State: Valid/)) { sub (" State:*","") sub ("^","\\&green ") } else { sub (" State:.*","") sub ("^","\\&red ") } } if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) { print "" } print if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) { print "----------------------------" } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[4. Disks status]------------------------------------------------------------- $navicli -h $SPIP2 getdisk -state -rb -bind > $BBTMP/$MYTMPFILE echo -e "\nDisks Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' BEGIN { FLAG=0 } { if (FLAG == 3) { FLAG=0 sub (" *","") sub ("Prct Bound: *","") while (sub ("[0123456789]+: +100 *","")) sub (" +$","%") bound=$0 if ((hotspare ~ /Enabled/) || (hotspare ~ /Ready/) || (hotspare ~ /Unbound/) || (hotspare ~ /Empty/)) { print status,"Disk",disk,hotspare
}
else {
if (hotspare ~ /Binding/) {
status="&green"
print status,"Disk",disk,hotspare,"LUN",bound
}
else {
status="&yellow"
print status,"Disk",disk,hotspare,"LUN",rebuild
print status,"Disk",disk," Hostspare=",hotspare,"LUN","
Rebuild=",rebuild
}
}
} if (FLAG == 2) { FLAG=3 sub (" *","") sub ("Prct Rebuilt: *","") while (sub ("[0123456789]+: +100 +","")) rebuild=$0 } if (FLAG == 1) { sub (" *","") sub ("State: *","") status=$0 FLAG=2 hotspare="" if ($0 ~ /Hot Spare/) { sub ("Hot Spare ","",status) hotspare="(Hot Spare)" } if (status ~ /Empty/) { hotspare=$status status="&clear" FLAG=3 } else { if ((status ~ /Enabled/) || (status ~ /Ready/) || (status ~ /Unbound/) || (status ~ /Binding/) || (status ~ /Expanding/)) { hotspare=$status status="&green" } else { if ((status ~ /Copying/) || (status ~ /Equalizing/)) { hotspare=$status status="&yellow" } else { hotspare=$status status="&red" } } } } if (($0 ~ /Bus +[0-9]+ +Enclosure +[0-9]+ +Disk +[0-9]+/)) { sub ("Bus +","") sub (" +Enclosure +","_") sub (" +Disk +","_") disk=$0 FLAG=1 } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[5. Trespass status]------------------------------------------------------------- $navicli -h $SPIP getlun -ismetalun -trespass -status > $BBTMP/$MYTMPFILE echo -e "\nTrespass Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /LOGICAL UNIT NUMBER/) { sub ("LOGICAL UNIT NUMBER ","") lun=$0
print "lun=",lun
}
else {
if ($0 ~ /Default Owner/) {
sub ("Default Owner: ","")
preffered_sp=$0
print "preferred sp="preffered_sp
}
else {
if ($0 ~ /Current owner/) {
sub ("Current owner: ","")
current_sp=$0
print "current_sp=",current_sp
status="&yellow"
if ( length(lun) < 4) {
print status " Lun " lun " -->\t Default
Owner:",preffered_sp," Current Owner:",current_sp
print status " Lun"
printf "%4s", lun
print " -->\t Default Owner:",preffered_sp," Current Owner:",current_sp
}
}
}
}
}' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[Check global color output]---
$CAT $BBTMP/ITC_CLARiiON.OUT.$$ | grep \&red > /dev/null 2>&1
ret=$?
if [ "$ret" = "0" ]; then
COLOR=red
else
$CAT $BBTMP/ITC_CLARiiON.OUT.$$ | grep \&yellow > /dev/null 2>&1
ret=$?
if [ "$ret" = "0" ]; then
COLOR=yellow
else
COLOR=green
fi
fi
#---[Generate information to send]---
LINE="status $EMC_ARRAY.$TEST $COLOR date
cat $BBTMP/ITC_CLARiiON.OUT.$$"
#if [ "$COLOR" != "red" ]; then $RM -f $BBTMP/ITC_CLARiiON.OUT.$$ $BBTMP/$MYTMPFILE #fi
#---[Send page info to BBDISPLAY]--- $BB $BBDISP "$LINE
$0 running on hostname"
#echo $BB $BBDISP "$LINE"
On Mon, Sep 17, 2012 at 11:37 PM, Vernon Everett <everett.vernon at gmail.com> wrote:
Hi all
Has anybody done any work on monitoring an EMC Clariion system, that they are prepared to share? I have no idea what I need to monitor yet, and we have nothing in place at the moment, so anything is better than what I have now.
All assistance appreciated.
Regards Vernon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
here exactly the same script that we use also but with minor tweaks we did.
you have to add a tag on the device configuration in hosts.cfg : emcstorage you can use emcstorage:EXCLUDE=[A0|A1|A2|B0|B2|B1] to exclude the port A0 A1 A2 B0 B2 B1 for instance.
#!/bin/bash
##--[special env]---- echo -n "Start " date
navicli=/opt/xymon/server/ext/emc/navicli # Where is the NAVICLI cmd GAWK=/bin/awk # need GNU AWK TEST="hardware" # The BB test name
##---[Check XYMONHOME is set]-------------------------------- if test "$XYMONHOME" = ""; then echo "XYMONHOME is not set... exiting"; exit 1 fi if [ ! -x $GAWK ]; then echo "This agent need GNU Awk, check GAWK variable"; exit 1 fi if test ! "$XYMONTMP"; then XYMONTMP=$XYMONHOME/tmp fi
xymongrep emcstorage* | while read ligne
do
SPIP=echo $ligne | awk '{print $1}'
EMC_ARRAY=echo $ligne | awk '{print $2}'
patch nli cf down (cf lines 117)
PORTS_EX=echo $ligne | grep EXCLUDE | sed "s/.*EXCLUDE=\[\(.*\)\]/\1/"
PORTS_EX=echo $PORTS_EX | sed 's/|/ /g'
TEST="hardware" # The XYMON test name MYTMPFILE=CLARIION_STATUS.$$ # Use a buffer tmp file for NAVICLI output XYMONPROG=XYMON_CLARiiON; export XYMONPROG # ???
#---[1. Get cache status]--------------------------------------------------------- $navicli -h $SPIP getcache > $XYMONTMP/$MYTMPFILE $CAT $XYMONTMP/$MYTMPFILE | $GAWK '
{
if ($0 ~ /SP Write|Read Cache State/) {
sub (" *"," ")
if ($0 ~ /State Enabled/) {
sub (" State Enabled","")
sub ("^","\\&green ")
} else {
sub (" State .*","")
sub ("^","\\&red ")
}
if ($0 !~ /SP Read Cache/) {
print
}
}
}' > $XYMONTMP/ITC_CLARiiON.CACHE.OUT.$$
#---[Check global color output]---
$CAT $XYMONTMP/ITC_CLARiiON.CACHE.OUT.$$ | grep \&red > /dev/null 2>&1
ret=$?
if [ "$ret" = "0" ]; then
COLOR=red
else
COLOR=green
fi
#---[Generate information to send]---
LINE="status $EMC_ARRAY.cache $COLOR date
cat $XYMONTMP/ITC_CLARiiON.CACHE.OUT.$$"
echo $EMC_ARRAY
cat $XYMONTMP/ITC_CLARiiON.CACHE.OUT.$$
$XYMON $BBDISP "$LINE"
rm $XYMONTMP/ITC_CLARiiON.CACHE.OUT.$$
#---[2. Port status]--------------------------------------------------------------
$navicli -h $SPIP port -list > $XYMONTMP/$MYTMPFILE
$CAT $XYMONTMP/$MYTMPFILE | $GAWK '
{
if ($0 ~ /Information about each SPPORT:/) {
FLAG=1
}
if (FLAG == 1) {
if (($0 ~ /SP Name/) || ($0 ~ /SP Port ID/) || ($0 ~ /Link Status/) || ($0 ~ /Port Status/)) {
sub (" *","")
if ($0 ~ /SP Name/) {
sub ("SP Name:","")
SP=$0
}
if ($0 ~ /SP Port ID/) {
sub ("SP Port ID:","Port ")
Port=$0
}
if ($0 ~ /Link Status/) {
if ($0 ~ /Status:Up/) {
sub ("Status:Up","\\&green")
} else {
sub ("Status:.*","\\&red")
}
LinkS=$0
}
if ($0 ~ /Port Status/) {
if ($0 ~ /Status:Online/) {
sub ("Status:Online","\\&green")
} else if ( $0 ~ /Status:DISABLED/ ) {
# port not activated, link down -> ok , gch 20080327
sub ("red","green",LinkS)
} else {
sub ("Status:.*","\\&red")
}
sub ("Port","Status")
PortS=$0
sub ("Status:","",PortS)
sub ("Status:","",LinkS)
print SP,Port,PortS,LinkS
}
}
}
}' > $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$
patch nli (ubber crado)
for ex in $PORTS_EX
do
SP=echo $ex | sed 's/\([A-Z]\).*/\1/'
PORT=echo $ex | sed 's/.*\([0-9]\)/\1/'
EXCLUDE="SP $SP Port $PORT"
grep -v "$EXCLUDE" $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$ > $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$.new
mv $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$.new $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$
done
#---[Check global color output]---
$CAT $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$ | grep \&red > /dev/null 2>&1
ret=$?
if [ "$ret" = "0" ]; then
COLOR=red
else
COLOR=green
fi
#---[Generate information to send]---
LINE="status $EMC_ARRAY.port $COLOR date
cat $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$"
$XYMON $BBDISP "$LINE"
rm $XYMONTMP/ITC_CLARiiON.PORT.OUT.$$
#---[3. DPE/DAE Status]-----------------------------------------------------------
$navicli -h $SPIP getcrus > $XYMONTMP/$MYTMPFILE
$CAT $XYMONTMP/$MYTMPFILE | $GAWK '
{
if (($0 ~ /[DPE|DAE] Bus . Enclosure ./) || ($0 ~ /State/)) {
if ($0 ~ /State/) {
sub ("Bus . Enclosure . ","")
sub (" *"," ")
sub ("Power","Power Supply")
if ($0 ~ /State: Valid/) {
sub ("Valid","Present")
}
if ($0 ~ /State: Present/) {
sub (" State: Present","")
sub ("^","\\&green ")
}
else {
sub (" State:.*","")
sub ("^","\\&red ")
}
}
if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) {
print ""
}
print
if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) {
print "----------------------------"
}
}
}' > $XYMONTMP/ITC_CLARiiON.HARD.OUT.$$
#---[Check global color output]---
$CAT $XYMONTMP/ITC_CLARiiON.HARD.OUT.$$ | grep \&red > /dev/null 2>&1
ret=$?
if [ "$ret" = "0" ]; then
COLOR=red
else
COLOR=green
fi
#---[Generate information to send]---
LINE="status $EMC_ARRAY.hardware $COLOR date
cat $XYMONTMP/ITC_CLARiiON.HARD.OUT.$$"
$XYMON $BBDISP "$LINE"
rm $XYMONTMP/ITC_CLARiiON.HARD.OUT.$$
#---[4. Disks status]------------------------------------------------------------- $navicli -h $SPIP getdisk -state > $XYMONTMP/$MYTMPFILE $CAT $XYMONTMP/$MYTMPFILE | $GAWK ' BEGIN { FLAG=0 } { if (FLAG == 1) { sub (" *","") sub ("State:","") status=$0 FLAG=0 hotspare="" if ($0 ~ /Hot Spare/) { sub ("Hot Spare ","",status) hotspare="(Hot Spare)" } if (status ~ /Empty/) { status="&clear" } else { if ((status ~ /Enabled/) || (status ~ /Ready/) || (status ~ /Unbound/) || (status ~ /Binding/)) { status="&green" } else { status="&red" } } print status,"Disk",disk,hotspare
if (FLAG == 1) { sub (" *","") sub ("State:","") status=$0 FLAG=0 hotspare="" if ($0 ~ /Hot Spare/) { sub ("Hot Spare ","",status) hotspare="(Hot Spare)" } if (status ~ /Empty/) { status="&clear" } else { if ((status ~ /Enabled/) || (status ~ /Ready/) || (status ~ /Unbound/) || (status ~ /Binding/)) { status="&green" } else { status="&red" } } print status,"Disk",disk,hotspare
}
if (($0 ~ /Bus +[0-9]+ +Enclosure +[0-9]+ +Disk +[0-9]+/) && (FLAG == 0)) {
sub ("Bus +","")
sub (" +Enclosure +","-")
sub (" +Disk +","-")
disk=$0
FLAG=1
}
}' > $XYMONTMP/ITC_CLARiiON.DISK.OUT.$$
#---[Check global color output]---
$CAT $XYMONTMP/ITC_CLARiiON.DISK.OUT.$$ | grep \&red > /dev/null 2>&1
ret=$?
if [ "$ret" = "0" ]; then
COLOR=red
else
COLOR=green
fi
#---[Generate information to send]---
LINE="status $EMC_ARRAY.disks $COLOR date
cat $XYMONTMP/ITC_CLARiiON.DISK.OUT.$$"
$XYMON $BBDISP "$LINE"
rm $XYMONTMP/ITC_CLARiiON.DISK.OUT.$$
rm $XYMONTMP/$MYTMPFILE done
Le 21 sept. 2012 à 17:15, Scott Pfister a écrit :
Here's a script we used to monitor clariion. It requires the navicli to be installed on the host that runs the monitoring script. In our case it was the xymon server. We also found it better to split the test accross both service processors. Hence the SPIP and SPIP2 variables. Ours had no NAS so we didn't montitor anything there. This was on the old deadcat site and we tweaked it a little.
#!/bin/sh #set -x #-------------------------------------------------
Big Brother - EMC CLARiiON FC-4700 Hardware Agent
This agent monitor the hardware of an EMC
CLARiiON FC-4700 Array using the Navisphere
Command Line Interface (NAVICLI)
Author: Sabey (sabey2000 at hotmail.com)
Feel free to improve or modify.
#==================================================
History
#--------------------------------------------------
Fri Feb 22 11:44:08 MET 2008
added global state yellow and Disk status
Notes:
If a disk failed, it was shown yellow in it`s list, but the global
status was unchanged. This is fixed now
The Disks will show now the status of a binding action in %
the same for replacement of a disk
Tue Aug 19 21:34:15 GMT 2003
Docs and fixes by N. Rettinghouse (rettinghouse.n0spam at att.net)
Notes:
GNU GAWK is needed. (www.sunfreeware.com for Solaris)
Copy clariion.sh into the bb/ext directory.
Add the following to your bb/etc/bb-bbexttab:
localhost : : clariion.sh
A restart of BigBro will be needed after adding bb-bbexttab.
Configure the variables shown below.
Add the BigBrother user to the /etc/Navisphere/agent.config file to
allow running of navicli.
Changed "HW" to "hw" to be consistant with other bb services.
Fixed conflict with $MACHINE BigBro internal variable.
Configure bb-hosts like this:
10.10.30.34 SAN-BigusDiskus # testip noping
#---------------------------------------------------
Thu Sep 4 11:57:48 EDT 2003
Docs and fixes by Sabey (sabey2000 at hotmail.com)
Notes:
Add support to CLARiiON CX600 (maybe all CX series...)
#############################################################################
Set these as needed.
#S#SPIP=172.25.2.239 # One of the two SP IP address #S#EMC_ARRAY="CLARiiON-1" # The name of your CLARiiON array SPIP= # One of the two SP IP address SPIP2= # One of the two SP IP address EMC_ARRAY="emcsan" # The name of your CLARiiON array navicli=/opt/Navisphere/bin/navicli # Where is the NAVICLI cmd GAWK=/usr/bin/awk # need GNU AWK ############################################################################# #############################################################################
TEST="emc" # The BB test name MYTMPFILE=CLARIION_STATUS.$$ # Use a buffer tmp file for NAVICLI output BBPROG=BB_CLARiiON; export BBPROG # ???
##---[Check BBHOME is set]-------------------------------- if test "$BBHOME" = ""; then echo "BBHOME is not set... exiting"; exit 1 fi if [ ! -x $GAWK ]; then echo "This agent need GNU Awk, check GAWK variable"; exit 1 fi
#---[Load bb global parameter]--------------------------- if test ! "$BBTMP"; then . $BBHOME/etc/bbdef.sh fi
#---[Collect DATA]--------------------------------------- echo "Status of ${EMC_ARRAY} Storage System" > $BBTMP/ITC_CLARiiON.OUT.$$ echo "============================================================================="
$BBTMP/ITC_CLARiiON.OUT.$$
#---[1. Get cache status]--------------------------------------------------------- $navicli -h $SPIP getcache > $BBTMP/$MYTMPFILE echo "SP Cache" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /SP Write|Read Cache State/) { sub (" *"," ") if ($0 ~ /State Enabled/) { sub (" State Enabled","") sub ("^","\\&green ") } else { sub (" State .*","") sub ("^","\\&red ") } if ($0 !~ /SP Read Cache/) { print } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[2. Port status]-------------------------------------------------------------- $navicli -h $SPIP2 port -list > $BBTMP/$MYTMPFILE echo -e "\nSP Link Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /Information about each SPPORT:/) { FLAG=1 } if (FLAG == 1) { if (($0 ~ /SP Name/) || ($0 ~ /SP Port ID/) || ($0 ~ /Link Status/) || ($0 ~ /Port Status/)) { sub (" *","") if ($0 ~ /SP Name/) { sub ("SP Name:","") SP=$0 } if ($0 ~ /SP Port ID/) { sub ("SP Port ID:","Port ") Port=$0 } if ($0 ~ /Link Status/) { if ($0 ~ /Status:Up/) { sub ("Status:Up","\\&green") } else { if ((Port == "Port 3") || (Port == "Port 2" )) { sub ("Status:.*","\\&clear") } else { sub ("Status:.*","\\&red") } } LinkS=$0 } if ($0 ~ /Port Status/) { if ($0 ~ /Status:Online/) { sub ("Status:Online","\\&green") } else { if ((Port == "Port 3") || (Port == "Port 2" )) { sub ("Status:.*","\\&clear") } else { sub ("Status:.*","\\&red") } } sub ("Port","Status") PortS=$0 sub ("Status:","",PortS) sub ("Status:","",LinkS) print SP,Port,PortS,LinkS } } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[3. DPE/DAE Status]----------------------------------------------------------- $navicli -h $SPIP getcrus > $BBTMP/$MYTMPFILE echo -e "\nDPE/DAE Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$
$CAT $BBTMP/$MYTMPFILE | $GAWK ' { if (($0 ~ /[DPE|DAE] Bus . Enclosure ./) || ($0 ~ /State/)) { if ($0 ~ /State/) { sub ("Bus . Enclosure . ","") sub (" *"," ") sub ("Power","Power Supply") if (($0 ~ /State: Present/) || ($0 ~ /State: Valid/)) { sub (" State:*","") sub ("^","\\&green ") } else { sub (" State:.*","") sub ("^","\\&red ") } } if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) { print "" } print if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) { print "----------------------------" } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[4. Disks status]------------------------------------------------------------- $navicli -h $SPIP2 getdisk -state -rb -bind > $BBTMP/$MYTMPFILE echo -e "\nDisks Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' BEGIN { FLAG=0 } { if (FLAG == 3) { FLAG=0 sub (" *","") sub ("Prct Bound: *","") while (sub ("[0123456789]+: +100 *","")) sub (" +$","%") bound=$0 if ((hotspare ~ /Enabled/) || (hotspare ~ /Ready/) || (hotspare ~ /Unbound/) || (hotspare ~ /Empty/)) { print status,"Disk",disk,hotspare
} else { if (hotspare ~ /Binding/) { status="&green" print status,"Disk",disk,hotspare,"LUN",bound
} else {status="&yellow" print status,"Disk",disk,hotspare,"LUN",rebuild
print status,"Disk",disk," Hostspare=",hotspare,"LUN","
Rebuild=",rebuild
}} } if (FLAG == 2) { FLAG=3 sub (" *","") sub ("Prct Rebuilt: *","") while (sub ("[0123456789]+: +100 +","")) rebuild=$0 } if (FLAG == 1) { sub (" *","") sub ("State: *","") status=$0 FLAG=2 hotspare="" if ($0 ~ /Hot Spare/) { sub ("Hot Spare ","",status) hotspare="(Hot Spare)" } if (status ~ /Empty/) { hotspare=$status status="&clear" FLAG=3 } else { if ((status ~ /Enabled/) || (status ~ /Ready/) || (status ~ /Unbound/) || (status ~ /Binding/) || (status ~ /Expanding/)) { hotspare=$status status="&green" } else { if ((status ~ /Copying/) || (status ~ /Equalizing/)) { hotspare=$status status="&yellow" } else { hotspare=$status status="&red" } } } } if (($0 ~ /Bus +[0-9]+ +Enclosure +[0-9]+ +Disk +[0-9]+/)) { sub ("Bus +","") sub (" +Enclosure +","_") sub (" +Disk +","_") disk=$0 FLAG=1 } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[5. Trespass status]------------------------------------------------------------- $navicli -h $SPIP getlun -ismetalun -trespass -status > $BBTMP/$MYTMPFILE echo -e "\nTrespass Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /LOGICAL UNIT NUMBER/) { sub ("LOGICAL UNIT NUMBER ","") lun=$0
print "lun=",lun
} else { if ($0 ~ /Default Owner/) { sub ("Default Owner: ","") preffered_sp=$0print "preferred sp="preffered_sp
} else { if ($0 ~ /Current owner/) { sub ("Current owner: ","") current_sp=$0print "current_sp=",current_sp
status="&yellow" if ( length(lun) < 4) { print status " Lun " lun " -->\t DefaultOwner:",preffered_sp," Current Owner:",current_sp
print status " Lun"
printf "%4s", lun
print " -->\t Default Owner:",preffered_sp," Current Owner:",current_sp
} } }}
}' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[Check global color output]--- $CAT $BBTMP/ITC_CLARiiON.OUT.$$ | grep \&red > /dev/null 2>&1 ret=$? if [ "$ret" = "0" ]; then COLOR=red else $CAT $BBTMP/ITC_CLARiiON.OUT.$$ | grep \&yellow > /dev/null 2>&1 ret=$? if [ "$ret" = "0" ]; then COLOR=yellow else COLOR=green fi fi #---[Generate information to send]--- LINE="status $EMC_ARRAY.$TEST $COLOR
datecat $BBTMP/ITC_CLARiiON.OUT.$$"#if [ "$COLOR" != "red" ]; then $RM -f $BBTMP/ITC_CLARiiON.OUT.$$ $BBTMP/$MYTMPFILE #fi
#---[Send page info to BBDISPLAY]--- $BB $BBDISP "$LINE
$0 running on
hostname" #echo $BB $BBDISP "$LINE"On Mon, Sep 17, 2012 at 11:37 PM, Vernon Everett <everett.vernon at gmail.com> wrote:
Hi all
Has anybody done any work on monitoring an EMC Clariion system, that they are prepared to share? I have no idea what I need to monitor yet, and we have nothing in place at the moment, so anything is better than what I have now.
All assistance appreciated.
Regards Vernon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
here another one only to graph IO disk activity by LUN:
#!/bin/sh
bbhostgrep emcio* | while read data;
do
HOST=echo $data | awk '{print $2}';
IP=echo $data | awk '{print $1}';
datafile="/tmp/.emcio-datafile-$HOST"
:> $datafile;
lunlist=`/opt/hobbit/server/ext/navicli -h $IP getlun |grep "LOGICAL UNIT NUMBER" | awk '{print $4}'`
for lun in $lunlist
do
/opt/hobbit/server/ext/navicli -h $IP getlun $lun > /tmp/.emc-io-$HOST-lun-$lun 2>/dev/null
if [ $? -eq 0 ]
then
NAME=`egrep ^Name /tmp/.emc-io-$HOST-lun-$lun |sed 's/Name//;s/ //g'`;
BLOCK_READ=`grep "Blocks read:" /tmp/.emc-io-$HOST-lun-$lun | head -n1 | awk '{print $3}'`;
BLOCK_WRITTEN=`grep "Blocks written:" /tmp/.emc-io-$HOST-lun-$lun | head -n 1 | awk '{print $3}'`;
echo $NAME $BLOCK_READ $BLOCK_WRITTEN
echo "[emciolun.$NAME.rrd]" >> $datafile
echo "DS:blocks_read:DERIVE:600:0:U $BLOCK_READ" >> $datafile
echo "DS:blocks_written:DERIVE:600:0:U $BLOCK_WRITTEN" >> $datafile
fi
done
echo "Publishing to hobbit."
$BB $BBDISP "data $HOST.trends
cat $datafile
"
done
[emciolun] FNPATTERN emciolun.(.*).rrd TITLE EMC Lun I/O YAXIS Bytes DEF:rb at RRDIDX@=@RRDFN@:blocks_read:AVERAGE DEF:wb at RRDIDX@=@RRDFN@:blocks_written:AVERAGE CDEF:r at RRDIDX@=rb at RRDIDX@,520,* CDEF:w at RRDIDX@=wb at RRDIDX@,520,* LINE1:r at RRDIDX@#@COLOR@:@RRDPARAM@ bytes/s Read -u 100 GPRINT:r at RRDIDX@:LAST: %5.1lf (cur) GPRINT:r at RRDIDX@:MAX: %5.1lf (max) GPRINT:r at RRDIDX@:MIN: %5.1lf (min) GPRINT:r at RRDIDX@:AVERAGE: %5.1lf (avg)\n LINE1:w at RRDIDX@#@COLOR@:@RRDPARAM@ bytes/s Written GPRINT:w at RRDIDX@:LAST: %5.1lf (cur) GPRINT:w at RRDIDX@:MAX: %5.1lf (max) GPRINT:w at RRDIDX@:MIN: %5.1lf (min) GPRINT:w at RRDIDX@:AVERAGE: %5.1lf (avg)\n
Le 21 sept. 2012 à 17:15, Scott Pfister a écrit :
Here's a script we used to monitor clariion. It requires the navicli to be installed on the host that runs the monitoring script. In our case it was the xymon server. We also found it better to split the test accross both service processors. Hence the SPIP and SPIP2 variables. Ours had no NAS so we didn't montitor anything there. This was on the old deadcat site and we tweaked it a little.
#!/bin/sh #set -x #-------------------------------------------------
Big Brother - EMC CLARiiON FC-4700 Hardware Agent
This agent monitor the hardware of an EMC
CLARiiON FC-4700 Array using the Navisphere
Command Line Interface (NAVICLI)
Author: Sabey (sabey2000 at hotmail.com)
Feel free to improve or modify.
#==================================================
History
#--------------------------------------------------
Fri Feb 22 11:44:08 MET 2008
added global state yellow and Disk status
Notes:
If a disk failed, it was shown yellow in it`s list, but the global
status was unchanged. This is fixed now
The Disks will show now the status of a binding action in %
the same for replacement of a disk
Tue Aug 19 21:34:15 GMT 2003
Docs and fixes by N. Rettinghouse (rettinghouse.n0spam at att.net)
Notes:
GNU GAWK is needed. (www.sunfreeware.com for Solaris)
Copy clariion.sh into the bb/ext directory.
Add the following to your bb/etc/bb-bbexttab:
localhost : : clariion.sh
A restart of BigBro will be needed after adding bb-bbexttab.
Configure the variables shown below.
Add the BigBrother user to the /etc/Navisphere/agent.config file to
allow running of navicli.
Changed "HW" to "hw" to be consistant with other bb services.
Fixed conflict with $MACHINE BigBro internal variable.
Configure bb-hosts like this:
10.10.30.34 SAN-BigusDiskus # testip noping
#---------------------------------------------------
Thu Sep 4 11:57:48 EDT 2003
Docs and fixes by Sabey (sabey2000 at hotmail.com)
Notes:
Add support to CLARiiON CX600 (maybe all CX series...)
#############################################################################
Set these as needed.
#S#SPIP=172.25.2.239 # One of the two SP IP address #S#EMC_ARRAY="CLARiiON-1" # The name of your CLARiiON array SPIP= # One of the two SP IP address SPIP2= # One of the two SP IP address EMC_ARRAY="emcsan" # The name of your CLARiiON array navicli=/opt/Navisphere/bin/navicli # Where is the NAVICLI cmd GAWK=/usr/bin/awk # need GNU AWK ############################################################################# #############################################################################
TEST="emc" # The BB test name MYTMPFILE=CLARIION_STATUS.$$ # Use a buffer tmp file for NAVICLI output BBPROG=BB_CLARiiON; export BBPROG # ???
##---[Check BBHOME is set]-------------------------------- if test "$BBHOME" = ""; then echo "BBHOME is not set... exiting"; exit 1 fi if [ ! -x $GAWK ]; then echo "This agent need GNU Awk, check GAWK variable"; exit 1 fi
#---[Load bb global parameter]--------------------------- if test ! "$BBTMP"; then . $BBHOME/etc/bbdef.sh fi
#---[Collect DATA]--------------------------------------- echo "Status of ${EMC_ARRAY} Storage System" > $BBTMP/ITC_CLARiiON.OUT.$$ echo "============================================================================="
$BBTMP/ITC_CLARiiON.OUT.$$
#---[1. Get cache status]--------------------------------------------------------- $navicli -h $SPIP getcache > $BBTMP/$MYTMPFILE echo "SP Cache" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /SP Write|Read Cache State/) { sub (" *"," ") if ($0 ~ /State Enabled/) { sub (" State Enabled","") sub ("^","\\&green ") } else { sub (" State .*","") sub ("^","\\&red ") } if ($0 !~ /SP Read Cache/) { print } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[2. Port status]-------------------------------------------------------------- $navicli -h $SPIP2 port -list > $BBTMP/$MYTMPFILE echo -e "\nSP Link Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /Information about each SPPORT:/) { FLAG=1 } if (FLAG == 1) { if (($0 ~ /SP Name/) || ($0 ~ /SP Port ID/) || ($0 ~ /Link Status/) || ($0 ~ /Port Status/)) { sub (" *","") if ($0 ~ /SP Name/) { sub ("SP Name:","") SP=$0 } if ($0 ~ /SP Port ID/) { sub ("SP Port ID:","Port ") Port=$0 } if ($0 ~ /Link Status/) { if ($0 ~ /Status:Up/) { sub ("Status:Up","\\&green") } else { if ((Port == "Port 3") || (Port == "Port 2" )) { sub ("Status:.*","\\&clear") } else { sub ("Status:.*","\\&red") } } LinkS=$0 } if ($0 ~ /Port Status/) { if ($0 ~ /Status:Online/) { sub ("Status:Online","\\&green") } else { if ((Port == "Port 3") || (Port == "Port 2" )) { sub ("Status:.*","\\&clear") } else { sub ("Status:.*","\\&red") } } sub ("Port","Status") PortS=$0 sub ("Status:","",PortS) sub ("Status:","",LinkS) print SP,Port,PortS,LinkS } } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[3. DPE/DAE Status]----------------------------------------------------------- $navicli -h $SPIP getcrus > $BBTMP/$MYTMPFILE echo -e "\nDPE/DAE Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$
$CAT $BBTMP/$MYTMPFILE | $GAWK ' { if (($0 ~ /[DPE|DAE] Bus . Enclosure ./) || ($0 ~ /State/)) { if ($0 ~ /State/) { sub ("Bus . Enclosure . ","") sub (" *"," ") sub ("Power","Power Supply") if (($0 ~ /State: Present/) || ($0 ~ /State: Valid/)) { sub (" State:*","") sub ("^","\\&green ") } else { sub (" State:.*","") sub ("^","\\&red ") } } if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) { print "" } print if ($0 ~ /[DPE|DAE] Bus . Enclosure ./) { print "----------------------------" } } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[4. Disks status]------------------------------------------------------------- $navicli -h $SPIP2 getdisk -state -rb -bind > $BBTMP/$MYTMPFILE echo -e "\nDisks Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' BEGIN { FLAG=0 } { if (FLAG == 3) { FLAG=0 sub (" *","") sub ("Prct Bound: *","") while (sub ("[0123456789]+: +100 *","")) sub (" +$","%") bound=$0 if ((hotspare ~ /Enabled/) || (hotspare ~ /Ready/) || (hotspare ~ /Unbound/) || (hotspare ~ /Empty/)) { print status,"Disk",disk,hotspare
} else { if (hotspare ~ /Binding/) {status="&green" print status,"Disk",disk,hotspare,"LUN",bound
} else {status="&yellow" print status,"Disk",disk,hotspare,"LUN",rebuild
print status,"Disk",disk," Hostspare=",hotspare,"LUN","
Rebuild=",rebuild
} }} if (FLAG == 2) { FLAG=3 sub (" *","") sub ("Prct Rebuilt: *","") while (sub ("[0123456789]+: +100 +","")) rebuild=$0 } if (FLAG == 1) { sub (" *","") sub ("State: *","") status=$0 FLAG=2 hotspare="" if ($0 ~ /Hot Spare/) { sub ("Hot Spare ","",status) hotspare="(Hot Spare)" } if (status ~ /Empty/) { hotspare=$status status="&clear" FLAG=3 } else { if ((status ~ /Enabled/) || (status ~ /Ready/) || (status ~ /Unbound/) || (status ~ /Binding/) || (status ~ /Expanding/)) { hotspare=$status status="&green" } else { if ((status ~ /Copying/) || (status ~ /Equalizing/)) { hotspare=$status status="&yellow" } else { hotspare=$status status="&red" } } } } if (($0 ~ /Bus +[0-9]+ +Enclosure +[0-9]+ +Disk +[0-9]+/)) { sub ("Bus +","") sub (" +Enclosure +","_") sub (" +Disk +","_") disk=$0 FLAG=1 } }' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[5. Trespass status]------------------------------------------------------------- $navicli -h $SPIP getlun -ismetalun -trespass -status > $BBTMP/$MYTMPFILE echo -e "\nTrespass Status" >> $BBTMP/ITC_CLARiiON.OUT.$$ echo "----------------------------" >> $BBTMP/ITC_CLARiiON.OUT.$$ $CAT $BBTMP/$MYTMPFILE | $GAWK ' { if ($0 ~ /LOGICAL UNIT NUMBER/) { sub ("LOGICAL UNIT NUMBER ","") lun=$0
print "lun=",lun
} else { if ($0 ~ /Default Owner/) { sub ("Default Owner: ","") preffered_sp=$0print "preferred sp="preffered_sp
} else { if ($0 ~ /Current owner/) { sub ("Current owner: ","") current_sp=$0print "current_sp=",current_sp
status="&yellow" if ( length(lun) < 4) { print status " Lun " lun " -->\t DefaultOwner:",preffered_sp," Current Owner:",current_sp
print status " Lun"
printf "%4s", lun
print " -->\t Default Owner:",preffered_sp," Current Owner:",current_sp
} } } }}' >> $BBTMP/ITC_CLARiiON.OUT.$$
#---[Check global color output]--- $CAT $BBTMP/ITC_CLARiiON.OUT.$$ | grep \&red > /dev/null 2>&1 ret=$? if [ "$ret" = "0" ]; then COLOR=red else $CAT $BBTMP/ITC_CLARiiON.OUT.$$ | grep \&yellow > /dev/null 2>&1 ret=$? if [ "$ret" = "0" ]; then COLOR=yellow else COLOR=green fi fi #---[Generate information to send]--- LINE="status $EMC_ARRAY.$TEST $COLOR
datecat $BBTMP/ITC_CLARiiON.OUT.$$"#if [ "$COLOR" != "red" ]; then $RM -f $BBTMP/ITC_CLARiiON.OUT.$$ $BBTMP/$MYTMPFILE #fi
#---[Send page info to BBDISPLAY]--- $BB $BBDISP "$LINE
$0 running on
hostname" #echo $BB $BBDISP "$LINE"On Mon, Sep 17, 2012 at 11:37 PM, Vernon Everett <everett.vernon at gmail.com> wrote:
Hi all
Has anybody done any work on monitoring an EMC Clariion system, that they are prepared to share? I have no idea what I need to monitor yet, and we have nothing in place at the moment, so anything is better than what I have now.
All assistance appreciated.
Regards Vernon
-- "Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
participants (4)
-
betsy.schwartz@gmail.com
-
everett.vernon@gmail.com
-
icepickjazz@gmail.com
-
nicolas@lienard.name