[hobbit] Sun Grid Engine monitoring

11 Nov 2008


      Nice, the update work's ... unfortunatley,
i've to change the command
qstat -s r
to
qhost -j -h
because i've got no output for qstat ... don't know why ..
.. for the mrtg- and graph-question i will be answer/ask again,
if i've got time test it ... :-(
martin
On Thu, 16 Oct 2008, Martin Flemming wrote:
...
Nice, that will be easier and quicker to test it, Thanks !
cheers,
martin
On Wed, 15 Oct 2008, rdeal wrote:
...
I just put an upgrade to the test on deadcat yesterday to version 0.3e...
Richard
...
From: Martin Flemming <martin.flemming at desy.de>
Reply-To: <hobbit at hswn.dk>
Date: Wed, 15 Oct 2008 22:58:46 +0200 (CEST)
To: <hobbit at hswn.dk>
Subject: Re: [hobbit] Sun Grid Engine monitoring
Thanks, i will test it and give feedback .. :-)
.. unfortunatley, i'm short in time for testing very short :-(
cheers,
martin
On Fri, 10 Oct 2008, rdeal wrote:
...
We have several queues, primary are default, fast, medium.
The config file is pretty simple:
Unscaled[_]: y
Target[sge_1]: /local/packages/IT/MRTG/bin/qgraph.sh both
MaxBytes[sge_1]: 625
AbsMax[sge_1]: 100000000
ShortLegend[sge_1]:  Jobs
YLegend[sge_1]: jobs
Options[sge_1]: absolute, gauge, noinfo, nopercent, growright,
integer,
unknaszero
Title[sge_1]: Current Jobs on JCVI GRID
Legend1[sge_1]: Running
Legend2[sge_1]: Queued
Legend3[sge_1]: Peak running
Legend4[sge_1]: Peak queued
LegendI[sge_1]: running
LegendO[sge_1]: queued
Colours[sge_1]: GREEN#00eb0c,BLUE#0000ff,GRAY#AAAAAA,VIOLET#ff00ff
PageTop[sge_1]: <H1>Current Jobs on JCVI GRID</H1>
<TABLE>
<TR><TD>Maintainer:</TD> <TD>rdeal at tigr.org</TD></TR>
<TR><TD>Description:</TD><TD>Number of current SGE Jobs running on
the
JCVI SGE GRID</TD></TR>
</TABLE>
routers.cgi*Options[sge_1]: noabsmax nodetail
Target[sge_7]: /local/packages/IT/MRTG/bin/qgraph.sh medium
MaxBytes[sge_7]: 354
AbsMax[sge_7]: 100000000
ShortLegend[sge_7]:  Jobs
YLegend[sge_7]: jobs
Options[sge_7]: absolute, gauge, noinfo, nopercent, growright,
integer,
unknaszero
Title[sge_7]: Current Jobs in medium.q and fast.q
Legend1[sge_7]: Medium
Legend2[sge_7]: Fast
Legend3[sge_7]: Peak medium
Legend4[sge_7]: Peak fast
LegendI[sge_7]: medium
LegendO[sge_7]: fast
Colours[sge_7]: GREEN#00eb0c,BLUE#0000ff,GRAY#AAAAAA,VIOLET#ff00ff
PageTop[sge_7]: <H1>Current Jobs on JCVI GRID</H1>
<TABLE>
<TR><TD>Maintainer:</TD> <TD>rdeal at tigr.org</TD></TR>
<TR><TD>Description:</TD><TD>Number of current SGE Jobs in the
fast.q and
medium.q</TD></TR>
</TABLE>
routers.cgi*Options[sge_7]: noabsmax nodetail
The script is rather simple:
!/bin/sh

case "$1" in
test )
echo 50
echo 20 ;;
total )
echo $T
echo 0 ;;
running )
echo $R
echo 0 ;;
msc )
T=/usr/local/bin/qstat -q fast.msc.q -s r | /usr/bin/tail +3 |  /usr/bin/wc -l
R=/usr/local/bin/qstat -q msc.q -s r | /usr/bin/tail +3 |   /usr/bin/wc  -l
echo $R
echo $T ;;
medium )
T=/usr/local/bin/qstat -q fast.q -s r | /usr/bin/tail +3 |   /usr/bin/wc  -l
R=/usr/local/bin/qstat -q medium.q -s r | /usr/bin/tail +3 |  /usr/bin/wc -l
echo $R
echo $T ;;
fast )
T=/usr/local/bin/qstat -q default.q -s r | /usr/bin/tail +3 |  /usr/bin/wc -l
R=/usr/local/bin/qstat -q fast.q -s r | /usr/bin/tail +3 |   /usr/bin/wc  -l
echo $R
echo $T ;;
desktop )
T=/usr/local/bin/qstat -q desktop.q -s r | /usr/bin/tail +3 |  /usr/bin/wc -l
R=/usr/local/bin/qstat -q fast.q -s r | /usr/bin/tail +3 |   /usr/bin/wc  -l
echo $R
echo $T ;;
jtc )
T=ssh wasp "use sgejcvi; qstat -g dt" | /usr/bin/tail +3 |   /usr/bin/wc  -l
R=ssh wasp "use sgejcvi; qstat -s r" | /usr/bin/tail +3 |   /usr/bin/wc  -l
echo $R
echo $T ;;

| both )
T=/usr/local/bin/qstat -g dt | /usr/bin/tail +3 | /usr/bin/wc -l
R=/usr/local/bin/qstat -s r | /usr/bin/tail +3 | /usr/bin/wc -l
echo $R
echo $T ;;
esac

echo foo
echo SGE
The two graphs configured above:
...
From: Martin Flemming <martin.flemming at desy.de>
Reply-To: <hobbit at hswn.dk>
Date: Fri, 10 Oct 2008 11:46:29 +0200 (CEST)
To: <hobbit at hswn.dk>
Subject: Re: [hobbit] Sun Grid Engine monitoring
Hmm, i didn't run mrtg ...
How can i do it for sge ?
.. i want to graph e.g jobs in the queue for each node, project or
queue ..
... everything i can do :-)
Or is it maybe possible to create graphs with the data of sge.sh
script ?
thanks & cheers
martin
On Fri, 10 Oct 2008, michael nemeth wrote:
...
Ok! well I run mrtg too so I can do that. Also I thought I try
creating
graphs for  the sge.sh when
it's implemented.   I've done custom graphs before (clearcase).
As I said I know little about SGE and the SGE support person knows
little
about hobbit.
Deal, Richard wrote:
>   What did you want to graph.  I use mrtg to graph global she
>   stats
>
>   Sent from a portable system with a small keyboard.
>
>   On Oct 9, 2008, at 5:51 PM, "Martin Flemming"
>   <martin.flemming at desy.de>
>   wrote:
>
> >
> >   Yep, i use it too ... but unfortunatley it's without trending
> >   graphs ..
> >
> >   Got anybody a solution for it ?
> >
> >   .. i'm very interested :-)
> >
> >   cheers,
> >      martin
> >
> >   On Thu, 9 Oct 2008, rdeal wrote:
> >
> > >
> > >   It is called sge.sh
> > >   http://www.deadcat.net/viewfile.php?fileid=920
> > >   And we run it under hobbit now.
> > >   Richard
> > >
> > > >   From: Galen Johnson <Galen.Johnson at sas.com>
> > > >   Reply-To: <hobbit at hswn.dk>
> > > >   Date: Thu, 9 Oct 2008 14:57:10 -0400
> > > >   To: "hobbit at hswn.dk" <hobbit at hswn.dk>
> > > >   Subject: RE: [hobbit] Sun Grid Engine monitoring
> > > >
> > > >   Where on deadcat?  I can't find anything that mentions it
> > > >   (a search
> > > >   on grid on
> > > >   Deacat turns up something for the California power grid).
> > > >   Most
> > > >   monitors from
> > > >   deadcat should work with little to no modification for
> > > >   Hobbit.
> > > >   Also, Hobbit
> > > >   will monitor whatever you tell it so the sky's the limit
> > > >   there...or
> > > >   in this
> > > >   case the scripter is the limit.
> > > >
> > > >   =G=
> > > >
> > > >   -----Original Message-----
> > > >   From: michael nemeth [mailto:michael.nemeth at lmco.com]
> > > >   Sent: Thursday, October 09, 2008 2:46 PM
> > > >   To: hobbit at hswn.dk
> > > >   Subject: [hobbit] Sun Grid Engine monitoring
> > > >
> > > >   While I not been involved in the effort yet,we are about
> > > >   to bring as
> > > >   "farm" under Sun Grid Engine in to production. Anyone
> > > >   doing any
> > > >   monitoring like this?  I saw a script on deadcat.  I
> > > >   really know
> > > >   little about it but said Id check into what monitoring
> > > >   and metrics
> > > >   could
> > > >   be done by hobbit.
> > > >
> > > >
> > > >   To unsubscribe from the hobbit list, send an e-mail to
> > > >   hobbit-unsubscribe at hswn.dk
> > > >
> > > >
> > > >
> > > >
> > > >   To unsubscribe from the hobbit list, send an e-mail to
> > > >   hobbit-unsubscribe at hswn.dk
> > > >
> > > >
> > >
> > >
> > >   To unsubscribe from the hobbit list, send an e-mail to
> > >   hobbit-unsubscribe at hswn.dk
> > >
> > >
> > >
> >
> >   Gruss
> >
> >         Martin Flemming
> >
> >
>> ______________________________________________________
> >   Martin Flemming
> >   DESY / IT          office : Building 2b / 008a
> >   Notkestr. 85       phone  : 040 - 8998 - 4667
> >   22603 Hamburg      mail   : martin.flemming at desy.de
>> ______________________________________________________
> >
> >   To unsubscribe from the hobbit list, send an e-mail to
> >   hobbit-unsubscribe at hswn.dk
> >
> >
>
>   To unsubscribe from the hobbit list, send an e-mail to
>   hobbit-unsubscribe at hswn.dk
>
>
>
To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
Gruss
     Martin Flemming

Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : 040 - 8998 - 4667
22603 Hamburg      mail   : martin.flemming at desy.de

To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
Gruss
     Martin Flemming

Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : 040 - 8998 - 4667
22603 Hamburg      mail   : martin.flemming at desy.de

To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
Gruss
  Martin Flemming

Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : 040 - 8998 - 4667
22603 Hamburg      mail   : martin.flemming at desy.de

To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk
Gruss
    Martin Flemming

Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : 040 - 8998 - 4667
22603 Hamburg      mail   : martin.flemming at desy.de

[hobbit] Sun Grid Engine monitoring

martin.flemming＠desy.de

!/bin/sh