Henrik,

here is a script which produces the following output on Tru64 V5.1B

hobbit at hifpxx31 $ ./memory.sh ... ... "status myhost,my,domaine.memory green Sun Aug 7 10:02:54 MEST 2005 Memory OK Memory Used Total Percentage &green Physical 1951M 2048M 95% &green Swap 1553M 8192M 18% "

--------------------------vvvvvvv-------------------------- #!/bin/ksh

################################################################################

Environment settings

The following two values could be overwritten by

$HCHOME/etc/common-def.sh

PERCENT_PHYS_WARN="100" # go yellow at this level PERCENT_PHYS_PANIC="101" # go red and page at this level PERCENT_SWAP_WARN="80" # go yellow at this level PERCENT_SWAP_PANIC="90" # go red and page at this level

MyP1=$1

MyFCT=memory

ScriptName=basename $0; export ScriptName

HCHOME=/usr/local/hobbit/client . $HCHOME/etc/common-def.sh

MEMDATA=$HCHOME/tmp/memdata.tmp.$$ SWAPDATA=$HCHOME/tmp/swapdata.tmp.$$

MyAGENT=hostname | sed "s/\./,/g" MyAGENT=echo ${MyAGENT}.${MyFCT}"

--

################################################################################

Functions

get_status() {

COLOR="green" STATUS="Memory OK"

/usr/bin/rm -f $MEMDATA /usr/bin/rm -f $SWAPDATA

initialize result values

PHYS_MEMORY=0 PHYS_MEMORY_USED=-1 SWAP_MEMORY=0 SWAP_MEMORY_USED=-1

/usr/bin/vmstat -P >$MEMDATA /sbin/swapon -s >$SWAPDATA

PHYS_MEMORY=cat $MEMDATA | grep "Total Physical Memory" | awk {'print $5'} | head -1 | cut -d. -f1 PHYS_MEMORY_KB=expr $PHYS_MEMORY \* 1024 PHYS_MEMORY_FREE_PAGES=cat $MEMDATA | grep "free pages" | awk {'print $4'} PHYS_MEMORY_FREE_KB=expr $PHYS_MEMORY_FREE_PAGES \* 8 PHYS_MEMORY_USED_KB=expr $PHYS_MEMORY_KB - $PHYS_MEMORY_FREE_KB PHYS_MEMORY_USED=expr $PHYS_MEMORY_USED_KB / 1024

SWAP_MEMORY_PAGES=cat $SWAPDATA | grep "Allocated space:" | awk {'print $3'} | tail -1 SWAP_MEMORY=expr $SWAP_MEMORY_PAGES \* 8 / 1024 SWAP_MEMORY_USED_PAGES=cat $SWAPDATA | grep "In-use space:" | awk {'print $3'} | tail -1 SWAP_MEMORY_USED=expr $SWAP_MEMORY_USED_PAGES \* 8 / 1024

if [ $PHYS_MEMORY -eq 0 ]; then exit 0 fi

percentage of memory used

PERCENT_PHYS_MEMORY_USED=0 PERCENT_SWAP_MEMORY_USED=0

if [ $PHYS_MEMORY -gt 0 ]; then PERCENT_PHYS_MEMORY_USED=expr 100 \* $PHYS_MEMORY_USED / $PHYS_MEMORY fi

if [ $SWAP_MEMORY -gt 0 ]; then PERCENT_SWAP_MEMORY_USED=expr 100 \* $SWAP_MEMORY_USED / $SWAP_MEMORY fi

calculate physical memory usage

if [ $PHYS_MEMORY_USED -ge 0 ]; then if [ $PERCENT_PHYS_MEMORY_USED -ge $PERCENT_PHYS_PANIC ]; then COLOR="red" PHYS_COLOR="red" STATUS="Memory **very** low" else if [ $PERCENT_PHYS_MEMORY_USED -ge $PERCENT_PHYS_WARN ]; then PHYS_COLOR="yellow" if [ "$COLOR" != "red" ]; then COLOR="yellow" STATUS="Memory low" fi else PHYS_COLOR="green" fi fi else PHYS_COLOR="clear" fi

virtual memory usage

if [ $SWAP_MEMORY_USED -ge 0 ]; then if [ $PERCENT_SWAP_MEMORY_USED -ge $PERCENT_SWAP_PANIC ]; then COLOR="red" SWAP_COLOR="red" STATUS="Memory **very** low" else if [ $PERCENT_SWAP_MEMORY_USED -ge $PERCENT_SWAP_WARN ]; then SWAP_COLOR="yellow" if [ "$COLOR" != "red" ]; then COLOR="yellow" STATUS="Memory low" fi else SWAP_COLOR="green" fi fi else STATUS="--" SWAP_COLOR="clear" fi

echo "$STATUS"

echo " Memory Used Total Percentage"

if [ ! "$PHYS_COLOR" = "clear" ]; then echo "&${PHYS_COLOR} Physical ${PHYS_MEMORY_USED}M ${PHYS_MEMORY}M ${PERCENT_PHYS_MEMORY_USED}%" |
awk '{printf "%s %-9s %8s %8s %12s\n",$1,$2,$3,$4,$5}' else echo "&${PHYS_COLOR} Unable to calculate virtual memory usage" fi

if [ ! "$SWAP_COLOR" = "clear" ]; then echo "&${SWAP_COLOR} Swap ${SWAP_MEMORY_USED}M ${SWAP_MEMORY}M ${PERCENT_SWAP_MEMORY_USED}%" |
awk '{printf "%s %-9s %8s %8s %12s\n",$1,$2,$3,$4,$5}' else echo "&${SWAP_COLOR} Unable to calculate virtual memory usage" fi

/usr/bin/rm -f $MEMDATA /usr/bin/rm -f $SWAPDATA

}

/usr/bin/rm -f $HCHOME/tmp/$MyAGENT touch $HCHOME/tmp/$MyAGENT

get_status > $HCHOME/tmp/$MyAGENT

MyDATE=/bin/date "+%Y-%m-%d %H:%M"

Send the result to all Hobbit display servers

debug version, do not send it ...

echo "... ... \"status $MyAGENT $COLOR date cat $HCHOME/tmp/$MyAGENT \""

#/usr/bin/rm -f $FDMDATA exit -------------------------------^^^^^^^^^^^^^^^^^^^^^^^-------------------

I use the following function to get processor load informations.

--------------------------vvvvvvvvvvvvvvvvvvvvvvvvvvvvv---------------------

get_status() {

COLOR="green"

set uptime | awk '{ gsub(/,/,""); printf "%s %s %s\n",$(NF-2),$(NF-1),$NF}'

AVG1=$1 AVG5=$2; export AVG5 AVG15=$3

NRPROC=ps -ef | wc -l | sed "s/ //g"; export NRPROC

The Hobbit "do_la.c" requires a line like "up: 36 days, 6 users, 239

procs, load=0.72"

uptime | awk '{printf "%s: %s %s %s %s %3s procs, load=%s\n",$2,$3,$4,$(NF-6),$(NF-5),ENVIRON["NRPROC"],ENVIRON["AVG5"]}'

set condition red yellow green

echo "LOAD average on hostname is:" echo "" echo " over the last 60 seconds : ${AVG1}" echo " over the last 5 minutes : ${AVG5}" echo " over the last 15 minutes : ${AVG15}"

COLOR=echo "$AVG5 $CPUWARN $CPUPANIC" | awk '{ $1 < $2 ? x="green" : $1 < $3 ? x="yellow" :x="red"; print (x)}'

}

--------------------------^^^^^^^^^^^^^^^^^^^^^^^^^--------------------------

Regards, Tony

Ok, here we are :

osadmin at test1> swapon -s Swap partition /dev/disk/dsk0b: Allocated space: 1572864 pages (12.00GB) In-use space: 51996 pages ( 3%) Free space: 1520868 pages ( 96%)

Total swap allocation: Allocated space: 1572864 pages (12.00GB) In-use space: 51996 pages ( 3%) Available space: 1520868 pages ( 96%) osadmin at test1 #

osadmin at test2> swapon -s Swap partition /dev/disk/dsk1b: Allocated space: 1048576 pages (8.00GB) In-use space: 199699 pages ( 19%) Free space: 848877 pages ( 80%)

Total swap allocation: Allocated space: 1048576 pages (8.00GB) In-use space: 199699 pages ( 19%) Available space: 848877 pages ( 80%) osadmin at test2>

osadmin at test3> swapon -s Swap partition /dev/disk/dsk3b: Allocated space: 868544 pages (6.63GB) In-use space: 8998 pages ( 1%) Free space: 859546 pages ( 98%)

Swap partition /dev/disk/dsk7c: Allocated space: 901440 pages (6.88GB) In-use space: 9136 pages ( 1%) Free space: 892304 pages ( 98%)

Total swap allocation: Allocated space: 1769984 pages (13.50GB) In-use space: 18134 pages ( 1%) Available space: 1751850 pages ( 98%) osadmin at test3>

osadmin at test4> swapon -s Swap partition /dev/disk/dsk3b: Allocated space: 868544 pages (6.63GB) In-use space: 126384 pages ( 14%) Free space: 742160 pages ( 85%)

Swap partition /dev/disk/dsk6c: Allocated space: 901440 pages (6.88GB) In-use space: 125968 pages ( 13%) Free space: 775472 pages ( 86%)

Swap partition /dev/disk/dsk7c: Allocated space: 901440 pages (6.88GB) In-use space: 125603 pages ( 13%) Free space: 775837 pages ( 86%)

Total swap allocation: Allocated space: 2671424 pages (20.38GB) In-use space: 377955 pages ( 14%) Available space: 2293469 pages ( 85%) osadmin at test4>

osadmin at test5> swapon -s Swap partition /dev/disk/dsk5b: Allocated space: 868544 pages (6.63GB) In-use space: 108247 pages ( 12%) Free space: 760297 pages ( 87%)

Swap partition /dev/disk/dsk8c: Allocated space: 901440 pages (6.88GB) In-use space: 108482 pages ( 12%) Free space: 792958 pages ( 87%)

Total swap allocation: Allocated space: 1769984 pages (13.50GB) In-use space: 216729 pages ( 12%) Available space: 1553255 pages ( 87%) osadmin at test5>

Regards, Tony

Thanks, but I'd really like to see what the "swapon -s" output looks like. This script works fine for a client that generates the status messages itself, but it's too heavy for the Hobbit client design.

(My idea with the Hobbit client is to have as little "intelligence" in the client as possible - I prefer to do very little processing of the data on the client, and just forward all of the raw output to the Hobbit server. That lets the admin see exactly what the system says about itself - sometimes these commands report things that don't show up in the interpreted data used for the Hobbit status messages. E.g. the "vmstat -P" output on OSF has a detailed break-down of how the memory is used, which might be useful when you get a memory-alert and need to know WHY all of the memory is gone. Instead of having to login to the box and run commands to figure it out, you can dig into the raw client-data which is available directly in Hobbit, and get the information you need. The fact that it also lets you centralize the alert configuration and potentially do some event correlation between different systems is just an added bonus).

Regards, Henrik

Henrik,

I have no access to Tru64 4.x systems... Here are some output examples from "uptime", I have implemented a simple client script which reports the uptime to the hobbit server, every day I had a blackout for 1 hour in de RRD. I have fixed it, the reason for that was:

date

Sun Aug 7 17:19:22 CEST 2005

uptime

17:19 up 90 days, 56 mins, 1 user, load average: 0.00, 0.01, 0.00

date

Sun Aug 7 17:23:58 CEST 2005

uptime

17:24 up 90 days, 1 hr, 1 user, load average: 0.00, 0.00, 0.00

date

Sun Aug 7 17:24:21 CEST 2005

uptime

17:24 up 90 days, 1:01, 1 user, load average: 0.02, 0.02, 0.01

==> There are 13 parameters, if uptime is some days + 1..59 mins (or 1 hr). ==> For the next 23 hours we have just 12 parameters.

Here the output on Solaris 8: $ uptime 5:30pm up 8 day(s), 16:42, 1 user, load average: 0.06, 0.02, 0.02

On Linux: $ uptime 7:30pm up 8 days, 17:20, 1 user, load average: 0.00, 0.00, 0.00

And here some outputs from "vmstat -P" from Compaq Tru64 UNIX V5.1B (Rev. 2650)

vmstat -P

Total Physical Memory = 6144.00 M = 786432 pages

Physical Memory Clusters:

start_pfn end_pfn type size_pages / size_bytes 0 885 pal 885 / 6.91M 885 786423 os 785538 / 6137.02M 786423 786432 pal 9 / 72.00k

Physical Memory Use:

start_pfn end_pfn type size_pages / size_bytes 885 1057 scavenge 172 / 1.34M 1057 2217 text 1160 / 9.06M 2217 2469 data 252 / 1.97M 2469 2963 bss 494 / 3.86M 2963 3257 kdebug 294 / 2.30M 3257 3264 cfgmgmt 7 / 56.00k 3264 3266 locks 2 / 16.00k 3266 3280 pmap 14 / 112.00k 3280 5374 unixtable 2094 / 16.36M 5374 5518 logs 144 / 1.12M 5518 23905 vmtables 18387 / 143.65M 23905 786423 managed 762518 / 5957.17M ============================ Total Physical Memory Use: 785538 / 6137.02M

Managed Pages Break Down:

   free pages = 18273
 active pages = 119391

inactive pages = 241854 wired pages = 82351 ubc pages = 300821 ================== Total = 762690

WIRED Pages Break Down:

vm wired pages = 11097 ubc wired pages = 0 meta data pages = 23494 malloc pages = 38790 contig pages = 3741 user ptepages = 3774 kernel ptepages = 789 free ptepages = 9 ================== Total = 81694

vmstat -P

Total Physical Memory = 2048.00 M = 262144 pages

Physical Memory Clusters:

start_pfn end_pfn type size_pages / size_bytes 0 396 pal 396 / 3.09M 396 262135 os 261739 / 2044.84M 262135 262144 pal 9 / 72.00k

Physical Memory Use:

start_pfn end_pfn type size_pages / size_bytes 396 545 scavenge 149 / 1.16M 545 1711 text 1166 / 9.11M 1711 1970 data 259 / 2.02M 1970 2464 bss 494 / 3.86M 2464 2757 kdebug 293 / 2.29M 2757 2764 cfgmgmt 7 / 56.00k 2764 2766 locks 2 / 16.00k 2766 2780 pmap 14 / 112.00k 2780 3979 unixtable 1199 / 9.37M 3979 4027 logs 48 / 384.00k 4027 10065 vmtables 6038 / 47.17M 10065 262135 managed 252070 / 1969.30M ============================ Total Physical Memory Use: 261739 / 2044.84M

Managed Pages Break Down:

   free pages = 14634
 active pages = 54190

inactive pages = 112275 wired pages = 48779 ubc pages = 22341 ================== Total = 252219

WIRED Pages Break Down:

vm wired pages = 9513 ubc wired pages = 0 meta data pages = 7780 malloc pages = 22728 contig pages = 3686 user ptepages = 4479 kernel ptepages = 272 free ptepages = 7 ================== Total = 48465

vmstat -P

Total Physical Memory = 4096.00 M = 524288 pages

Physical Memory Clusters:

start_pfn end_pfn type size_pages / size_bytes 0 397 pal 397 / 3.10M 397 524279 os 523882 / 4092.83M 524279 524288 pal 9 / 72.00k

Physical Memory Use:

start_pfn end_pfn type size_pages / size_bytes 397 545 scavenge 148 / 1.16M 545 1436 text 891 / 6.96M 1436 1612 data 176 / 1.38M 1612 1845 bss 233 / 1.82M 1845 2062 kdebug 217 / 1.70M 2062 2069 cfgmgmt 7 / 56.00k 2069 2071 locks 2 / 16.00k 2071 2085 pmap 14 / 112.00k 2085 3745 unixtable 1660 / 12.97M 3745 3841 logs 96 / 768.00k 3841 16012 vmtables 12171 / 95.09M 16012 524279 managed 508267 / 3970.84M ============================ Total Physical Memory Use: 523882 / 4092.83M

Managed Pages Break Down:

   free pages = 159031
 active pages = 94097

inactive pages = 186424 wired pages = 43159 ubc pages = 25704 ================== Total = 508415

WIRED Pages Break Down:

vm wired pages = 8260 ubc wired pages = 0 meta data pages = 15665 malloc pages = 10142 contig pages = 3750 user ptepages = 4808 kernel ptepages = 526 free ptepages = 8 ================== Total = 43159

Regards, Tony

David Gore wrote: Unfortunately we are on Tru64 4.x

Hi,

Henrik Stoerner schrieb:

(except for the netstat data, will do that tomorrow). If someone could confirm that AIX, HP-UX and OSF/1 work (with whatever version of those you have), it would be nice.

I just tried to compile the snapshot under AIX 5.1 ML08 using gcc and gmake. The compilation stops with the following error:

In file included from /home/hobbit/snapshot/include/libbbgen.h:24, from sendmsg.c:35: /home/hobbit/snapshot/include/../lib/osdefs.h:21: conflicting types for socklen_t' /usr/include/sys/socket.h:80: previous declaration of socklen_t' sendmsg.c: In function sendtobbd': sendmsg.c:322: warning: passing arg 5 of getsockopt' from incompatible pointer type make[1]: *** [sendmsg.o] Error 1 make[1]: Leaving directory `/home/hobbit/snapshot/lib' make: *** [lib-client] Error 2

Dirk

Hi Henrik,

Could you send me the contents of the "include/config.h" file ?

Good news, the compilation succeeded. I had an old version of gcc installed when I did the first compilation. After the update I called "make clean", deleted the Makefile and called configure and make again, but the config.h hasn't been rebuild. I just deleted the file manually and now the compilation works.

Best wishes, Dirk

On Tue, Aug 09, 2005 at 10:26:16AM -0400, Michael Nemeth wrote:

OK with the latest snapshot 09-Aug-2005 Im able to compile on: Solaris 2.6 with no issues, hp11 with no issues except changing CC=cc to CC=gcc

Great.

hp1020 issue: the -lnsl flag , library not found. I tried -lresolv no luck . I removed both and it compiled; is this OK does the client use this?

If it compiles OK, then it isn't needed. All library dependencies are resolved at compile/link time.

At this time I cannot test if they actually work except for hpux 11.

With hpux 11 client two issues: the network I/O graph stopped working . Stop working after a snapshot from last week. Looks like it worked with 4.1.1. I tried stopping hobbit and removing the rrd with no luck.

Please send me a copy of the client/tmp/msg.txt from this client. I did some work on the netstat parser, so it is quite possible that I broke something - having a sample of the input data makes testing a lot easier.

Next is not a big issue yet (it may/should be on my Classed side where clearcase runs) Disk is picking automounts: /sw1/usr/local 4183116 4010394 172722 96% /usr/local this is an automount

OK, I need your expertise here: As you can see in the hobbiclient-hpux.sh script, it runs the command "df -Pk" to get the df data. What options can be added to the df command line to avoid picking up these mounts ?

Regards, Henrik

On Tue, Aug 09, 2005 at 02:31:21PM -0400, Michael Nemeth wrote:

As far a df goes there are really no options to do this You can look for fs types but there are several valid ones. In BB I just grepped df -Pk | grep ^/dev

Well, you can do that in the Hobbit client as well. The ~hobbit/client/bin/hobbitclient-hp-ux.sh client program *is* a shell script, so sticking your grep on the df command there will work.

I'll look at your netstat output.

Henrik

On Thu, Aug 11, 2005 at 07:13:09AM -0400, Michael Nemeth wrote:

One other thing , one other thing , a popular Distro of Perl for the hp goes in /opt/perl but others go in /usr and /usr/local and the old one in /usr/contrib. Ive all of them on system . The config script pick one I don't want to use; so Ive been edit the perl.sh just to have /opt/perl in it. Be nice if I could just supply this as an option the configure.

The perl check is actually a left-over from the old perl-based maint.pl script. None of Hobbit requires Perl anymore.

So I've just deleted that check from the configure script.

Henrik