- Xymon - Xymon mailman web

[hobbit] Some thoughts about alerts, acks and escalations
by Larry.Barber＠usda.gov 13 Apr '05

13 Apr '05

Yes, very useful. I think the biggest flaw with BigBrother is the way one red alert can mask subsequent red alerts, even alerts from different machines. There needs to be a way for the operations people to clear an alert after they've noticed it and logged it. Thanks, Larry Barber On Wed, 2005-04-13 at 00:49 -0500, henrik at hswn.dk wrote: > I'm beginning to look at the issue of escalating alerts. And I've had > an idea that I'd like to get some feedback on before I go ahead and > implement it. > > Right now, Hobbit doesn't handle escalating an alert. When someone > receives an alert message, they can ack it - when they do, all alerts > stop and the item disappears from the "Critical systems" page (the NK > page). > > BB has the concept of escalating an alert, meaning that some > recipients of an alert will get the alert message even if the alert > has been acknowledged. > > > What I'd like to have is the BB system with a finer granularity. A > recipient in the hobbit-alerts.cfg file has an associated "level", > default is 1. > > I want our NOC guys who do nothing but stare at the NK page 24x7 to > be > able to acknowledge an alert - and that just gets it off their > monitor, it doesn't stop alerts from going out. A "level 0" > acknowledgment - this is just to log that a trouble ticket has been > raised for the issue. > > A technician (who is a "level 1" recipient) can acknowledge the alert > he receives - this will stop alert messages from going out to other > "level 1" receipients, so all of the engineers can concentrate on > doing what needs to be done. > > Alerts will still be sent to recipients who are "level 2" and above - > these are the equivalent of the BB "escalation" alerts. They can ack > the alert if they'd like to turn off more alert messages, of course. > > You can have even higher levels if you like, probably going up the > hierarchy of managers. I don't think we'll using more than the 3 > levels I've described, but there is no reason to impose any limit. > > > Does that sound like it would be useful? > > > Regards, > Henrik > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > > >

1 0

[hobbit] Some thoughts about alerts, acks and escalations
by mlowery＠alliedtechgroup.com 13 Apr '05

13 Apr '05

Absolutely! I'm currently the level 3 guy, but if one of the level 1 guys ack the alert, I don't get it anymore... Sounds like a great idea! Michael -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Wednesday, April 13, 2005 12:49 AM To: hobbit at hswn.dk Subject: [hobbit] Some thoughts about alerts, acks and escalations I'm beginning to look at the issue of escalating alerts. And I've had an idea that I'd like to get some feedback on before I go ahead and implement it. Right now, Hobbit doesn't handle escalating an alert. When someone receives an alert message, they can ack it - when they do, all alerts stop and the item disappears from the "Critical systems" page (the NK page). BB has the concept of escalating an alert, meaning that some recipients of an alert will get the alert message even if the alert has been acknowledged. What I'd like to have is the BB system with a finer granularity. A recipient in the hobbit-alerts.cfg file has an associated "level", default is 1. I want our NOC guys who do nothing but stare at the NK page 24x7 to be able to acknowledge an alert - and that just gets it off their monitor, it doesn't stop alerts from going out. A "level 0" acknowledgment - this is just to log that a trouble ticket has been raised for the issue. A technician (who is a "level 1" recipient) can acknowledge the alert he receives - this will stop alert messages from going out to other "level 1" receipients, so all of the engineers can concentrate on doing what needs to be done. Alerts will still be sent to recipients who are "level 2" and above - these are the equivalent of the BB "escalation" alerts. They can ack the alert if they'd like to turn off more alert messages, of course. You can have even higher levels if you like, probably going up the hierarchy of managers. I don't think we'll using more than the 3 levels I've described, but there is no reason to impose any limit. Does that sound like it would be useful? Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

[hobbit] Some thoughts about alerts, acks and escalations
by tbarnes1＠hfhs.org 13 Apr '05

13 Apr '05

YES - that would work well here. Terry Barnes Siemens Com @ HFHS 248-853-4968 (Office) 586-405-8382 (Cellular) 248-844-3030 (Fax) 5864058382 at messaging.nextel.com (Text Pager) tbarnes1 at hfhs.org >>> henrik at hswn.dk 4/13/05 1:49:07 AM >>> I'm beginning to look at the issue of escalating alerts. And I've had an idea that I'd like to get some feedback on before I go ahead and implement it. Right now, Hobbit doesn't handle escalating an alert. When someone receives an alert message, they can ack it - when they do, all alerts stop and the item disappears from the "Critical systems" page (the NK page). BB has the concept of escalating an alert, meaning that some recipients of an alert will get the alert message even if the alert has been acknowledged. What I'd like to have is the BB system with a finer granularity. A recipient in the hobbit-alerts.cfg file has an associated "level", default is 1. I want our NOC guys who do nothing but stare at the NK page 24x7 to be able to acknowledge an alert - and that just gets it off their monitor, it doesn't stop alerts from going out. A "level 0" acknowledgment - this is just to log that a trouble ticket has been raised for the issue. A technician (who is a "level 1" recipient) can acknowledge the alert he receives - this will stop alert messages from going out to other "level 1" receipients, so all of the engineers can concentrate on doing what needs to be done. Alerts will still be sent to recipients who are "level 2" and above - these are the equivalent of the BB "escalation" alerts. They can ack the alert if they'd like to turn off more alert messages, of course. You can have even higher levels if you like, probably going up the hierarchy of managers. I don't think we'll using more than the 3 levels I've described, but there is no reason to impose any limit. Does that sound like it would be useful? Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk ============================================================================== HFHS CONFIDENTIALITY NOTICE: This email contains information from the sender that may be CONFIDENTIAL, LEGALLY PRIVILEGED, PROPRIETARY or otherwise protected from disclosure. This email is intended for use only by the person or entity to whom it is addressed. If you are not the intended recipient, any use, disclosure, copying, distribution, printing, or any action taken in reliance on the contents of this email, is strictly prohibited. If you received this email in error, please contact the sending party by replying in an email to the sender, delete the email from your computer system and shred any paper copies of the email you printed. Note to Patients: There are a number of risks you should consider before using e-mail to communicate with us. These risks are described in our Privacy Policy at http://henryford.com. Review that policy carefully before continuing to communicate with us by e-mail. For greater Internet security, our policy describes the Henry Ford MyHealth electronic communication process - you may register at http://henryford.com. If you do not believe that our policy gives you the privacy and security protection you need, do not send e-mail or Internet communications to us. ==============================================================================

1 0

MRTG & Hobbit
by mlowery＠alliedtechgroup.com 12 Apr '05

12 Apr '05

This may not be directly Hobbit related, but... I get the message "Time stamp in log is on the future !" on each of my MRTG pages. I was not getting this when running bigbrother, so I'm wondering what would cause this or if there is something I can do to keep that from appearing. MRTG is working fine, just this annoying message. Thanks, Michael

3 2

Windows Client
by slambert＠lambertsolutions.com 12 Apr '05

12 Apr '05

Hello all, I'm trying to create a hobbit client for windows. Can someone tell me how the info is to be sent? Possibly a format. Should it be sent via a file or straight connection. Any ideas would be great. Thanks, Stew P.S. I know this has been said before, but.. Many, Many, Many THANKS for all your work Henrik

2 2

[hobbit] acks & maint.pl aren't working
by dstuffle＠deltafaucet.com 12 Apr '05

12 Apr '05

Tom Georgoulias wrote: > Upgraded to 4.0.2 this morning and while most everything > went smoothly, a couple of things did not. Not sure if > these two are > related just yet. > > ack: > > Cannot acknowledge an alert using the cookie from either > the subject line of the email alert or the command line > (~/server/bin/bb 127.0.0.1 "hobbitdboard > host=secret.nandomedia.com test=conn > fields=hostname,testname,cookie"). > > Strangely enough, it seems teh cookies are still being > regenerated frequently, instead of remaining valid for 24 > hours (like I > thought the > latest patch did). > > [bb at radm202p ext]$ ~/server/bin/bb 127.0.0.1 "hobbitdboard > host=secret.nandomedia.com test=conn > fields=hostname,testname,cookie" > secret.nandomedia.com|conn|402332 [bb at radm202p ext]$ > ~/server/bin/bb 127.0.0.1 "hobbitdack 402332 30 using > command line ack for 402332" > [bb at radm202p ext]$ tail ~/data/acks/acklog > [bb at radm202p ext]$ tail ~/log/hobbitd.log > 2005-04-11 09:05:24 How did this happen? clients=0, > s.sem_op=0 2005-04-11 09:05:34 Setup complete 2005-04-11 > 09:08:28 How did this happen? clients=0, s.sem_op=0 > 2005-04-11 09:08:38 Setup complete 2005-04-11 09:40:34 > Cookie 87973 not found, dropping ack 2005-04-11 09:41:38 > Cookie 587666 not found, dropping ack 2005-04-11 09:54:35 > Cookie 587666 not found, dropping ack 2005-04-11 09:55:45 > Cookie 873795 not found, dropping ack 2005-04-11 09:56:33 > Cookie 873795 not found, dropping ack 2005-04-11 10:06:29 > Cookie 402332 not found, dropping ack [bb at radm202p ext]$ > > Trying to disable the same system via the disable CGI > didn't work either. The script stops at "Processing" and > the system is > not disabled. > > This is what I see in /var/log/httpd/error_log > > [Mon Apr 11 09:59:25 2005] [error] [client 10.1.1.20] > [Mon Apr 11 09:59:25 2005] maint.pl: Use of uninitialized > value in pattern match (m//) at > /home/bb/cgi-secure/maint.pl line 222. > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk My acks aren't working either. Not through ack menu pick or email reply's. I'm getting the same "maint.pl: Use of uninitialized value..." error, but the times don't correspond with my ack attempts. I do see the acks getting logged in the procmail.log file, but they're not ack'ing. This was happening before and after the 4.0.2 upgrade. ~~~~~~~~~~~~~~ David Stuffle Delta Faucet Company This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Finally, the recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

3 4

[hobbit] MRTG & Hobbit
by mlowery＠alliedtechgroup.com 12 Apr '05

12 Apr '05

Great! Thanks for taking the time to look at that for me! Michael -----Original Message----- From: Henrik Storner [mailto:henrik at hswn.dk] Sent: Tuesday, April 12, 2005 9:51 AM To: hobbit at hswn.dk Subject: Re: [hobbit] MRTG & Hobbit In <40534FFD21F8E9428DD8CCF2B60B09EC6E91FE at atlitdc01.atg.local> "Lowery, Michael" <mlowery at alliedtechgroup.com> writes: >Bbmrtg.sh has this in it, but I don't know how to determine what it's >doing and why it's reporting the error. > > timenow=`$BBHOME/bin/touchtime -e` > timediff=`echo "$lasttimestamp $timenow" | $AWK '{ print $2 - $1}'` [snip] > if [ "$timediff" -gt 0 ] > then [cut] > else > ERRMSG="Time stamp in log is on the future !" > fi I suspect this may happen if you run bbmrtg with BBHOME pointing to the Hobbit installation - because the "touchtime" utility does not exist. So you get "timediff" to be negative from the awk command. If you have GNU date installed, I'd just replace that line with timenow=`date +%s` and see if that resolves it. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

[hobbit] MRTG & Hobbit
by mlowery＠alliedtechgroup.com 12 Apr '05

12 Apr '05

Bbmrtg.sh has this in it, but I don't know how to determine what it's doing and why it's reporting the error. # If MRTG hasn't updated the file in $BBSLEEP seconds then # don't do anything as to possibly have BBDISPLAY generate a purple # status at one point if MRTG doesn't update the log in 30 minutes. timenow=`$BBHOME/bin/touchtime -e` timediff=`echo "$lasttimestamp $timenow" | $AWK '{ print $2 - $1 }'` if [ "$timediff" -gt 0 ] then # Do we have a stale MRTG .log file ? No update ... # Will provoke a purple after a while if [ "$timediff" -gt "$MRTGSTALESECS" -a "$PURPLECHECK" = "TRUE" ] then continue fi ERRMSG="" if [ "$currin" -ge "$redlevelinmax" -o "$currout" -ge "$redleveloutmax" ] then COLOR="red" elif [ "$currin" -lt "$redlevelinmin" -o "$currout" -lt "$redleveloutmin" ] then COLOR="red" elif [ "$currin" -ge "$yellowlevelinmax" -o "$currout" -ge "$yellowleveloutmax" ] then COLOR="yellow" elif [ "$currin" -lt "$yellowlevelinmin" -o "$currout" -lt "$yellowleveloutmin" ] then COLOR="yellow" if [ "$timediff" -gt 0 ] then # Do we have a stale MRTG .log file ? No update ... # Will provoke a purple after a while if [ "$timediff" -gt "$MRTGSTALESECS" -a "$PURPLECHECK" = "TRUE" ] then continue fi ERRMSG="" if [ "$currin" -ge "$redlevelinmax" -o "$currout" -ge "$redleveloutmax" ] then COLOR="red" elif [ "$currin" -lt "$redlevelinmin" -o "$currout" -lt "$redleveloutmin" ] then COLOR="red" elif [ "$currin" -ge "$yellowlevelinmax" -o "$currout" -ge "$yellowleveloutmax" ] then COLOR="yellow" elif [ "$currin" -lt "$yellowlevelinmin" -o "$currout" -lt "$yellowleveloutmin" ] then COLOR="yellow" fi else ERRMSG="Time stamp in log is on the future !" fi rm -f $BBTMP/BBMRTG.$$ -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Tuesday, April 12, 2005 9:18 AM To: hobbit at hswn.dk Subject: Re: [hobbit] MRTG & Hobbit On Tue, Apr 12, 2005 at 09:11:16AM -0500, Lowery, Michael wrote: > This may not be directly Hobbit related, but... > > I get the message "Time stamp in log is on the future !" on each of my > MRTG pages. I was not getting this when running bigbrother, so I'm > wondering what would cause this or if there is something I can do to > keep that from appearing. MRTG is working fine, just this annoying > message. No idea, really. It would have been nice to know *which* logfile holds this futuristic timestamp. Any mention of that ? Any output from the mrtg cron-job ? Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

2 1

acks & maint.pl aren't working
by tgeorgoulias＠mcclatchy.com 11 Apr '05

11 Apr '05

Upgraded to 4.0.2 this morning and while most everything went smoothly, a couple of things did not. Not sure if these two are related just yet. ack: Cannot acknowledge an alert using the cookie from either the subject line of the email alert or the command line (~/server/bin/bb 127.0.0.1 "hobbitdboard host=secret.nandomedia.com test=conn fields=hostname,testname,cookie"). Strangely enough, it seems teh cookies are still being regenerated frequently, instead of remaining valid for 24 hours (like I thought the latest patch did). [bb at radm202p ext]$ ~/server/bin/bb 127.0.0.1 "hobbitdboard host=secret.nandomedia.com test=conn fields=hostname,testname,cookie" secret.nandomedia.com|conn|402332 [bb at radm202p ext]$ ~/server/bin/bb 127.0.0.1 "hobbitdack 402332 30 using command line ack for 402332" [bb at radm202p ext]$ tail ~/data/acks/acklog [bb at radm202p ext]$ tail ~/log/hobbitd.log 2005-04-11 09:05:24 How did this happen? clients=0, s.sem_op=0 2005-04-11 09:05:34 Setup complete 2005-04-11 09:08:28 How did this happen? clients=0, s.sem_op=0 2005-04-11 09:08:38 Setup complete 2005-04-11 09:40:34 Cookie 87973 not found, dropping ack 2005-04-11 09:41:38 Cookie 587666 not found, dropping ack 2005-04-11 09:54:35 Cookie 587666 not found, dropping ack 2005-04-11 09:55:45 Cookie 873795 not found, dropping ack 2005-04-11 09:56:33 Cookie 873795 not found, dropping ack 2005-04-11 10:06:29 Cookie 402332 not found, dropping ack [bb at radm202p ext]$ Trying to disable the same system via the disable CGI didn't work either. The script stops at "Processing" and the system is not disabled. This is what I see in /var/log/httpd/error_log [Mon Apr 11 09:59:25 2005] [error] [client 10.1.1.20] [Mon Apr 11 09:59:25 2005] maint.pl: Use of uninitialized value in pattern match (m//) at /home/bb/cgi-secure/maint.pl line 222.

2 4

hobbit & larrd
by jonbjorn＠hi.is 11 Apr '05

11 Apr '05

Hi hobbits. Is it possible to have larrd display weekdays in other languages ? -- Jon Bjorn Njalsson <jonbjorn at hi.is>

2 1