[hobbit] Hobbit not recognizing certain tests?
-----Original Message----- From: FARRIOR, Andy [mailto:Andy.Farrior at victoriacollege.edu] Sent: Monday, July 16, 2007 5:30 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Hobbit not recognizing certain tests?
Andy, wonderful to hear from you. And thanks for your method of getting snmp traps into Hobbit. It is a bit of a feather in my cap that I can do so. My boss is anxious to get rid of our highly priced NetCool monitoring package, but we needed to be able to monitor traps, at least from our F5 devices.
I haven't modified trap.pl in a very *long* time. Hmm... (I'm still using it.)
Going back over the code, I'm running hobbitdboard so I can get the validtime field for the test: "$BB $BBDISP \"hobbitdboard test=trap fields=hostname,validtime,color\" "
In case there's a host that doesn't send a trap that often, I wanted to change the color to green instead of having it go purple and generate an alert. So to get the time left for a test, I needed the hobbitdboard results. A side-effect is that the column won't appear until the device sends it's first trap.
Yes, I looked again at the code and realized that I was mistaken, and have already apologized to Henrik for having the temerity to suggest that there might be a bug in his code.
For the problem device, is the log from SEC showing a hostname or IP address for the device sending the trap?
Yes, but that was sometime ago. Maybe before I made some other corrections to get things working. I'm tempted to just down a server to cause a trap.
There may be a problem if it's sending a status message to Hobbit with an IP address instead of a hostname. I don't think trap.pl does a good job (or at all) of trying to resolve an IP address to a hostname that's in bb-hosts.
Yes, I've seen that. In order to get things to work, I've had to make /etc/host file entries and make sure that the sending devices are configured to send their traps via certain interfaces so that the IP address will correspond to the host file address.
Hope that helps, Andy
-----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Monday, July 16, 2007 3:41 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Hobbit not recognizing certain tests?
On Mon, Jul 16, 2007 at 12:43:55PM -0400, Eric Jacobs wrote:
The trap.pl uses hobbitboard to find all the hosts that have the "trap" test defined.
Bad idea - there's a chicken-and-egg problem here.
Columns do not appear in the hobbitdboard output until they exist, i.e. a status has been reported to Hobbit. So if *something* must report a "trap" status to Hobbit before the trap.pl script sees that it should do the "trap" thing ... well, I guess you can tell where I'm going.
Scripts that implement custom tests should use the "bbhostgrep" utility to scan the bb-hosts file for the hosts that have a specific tag defined.
There is actually another way it can be done, but it's not yet documented: You can send a "hostinfo" command to hobbitd, and it will return a pre-parsed version of the bb-hosts file - one line per hosts, with the fields delimited by '|' characters, eg: mail.hswn.dk|172.16.10.2|NET:intern|COMMENT:Internal mail server|smtp|pop3|imap
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Being familiar with both Netcool and Hobbit, I would never consider replacing Netcool trap management with Hobbit. Netcool processes traps in "near real time" and scales quite large. Hobbit does not provide real time monitoring (default is 5 minute samples, with 1 minute screen updates). The only scenario where Hobbit might successfully replace a Netcool system is if you only want to know that *something bad* happened in the last five minutes, and you don't mind drilling down into a Hobbit screen to find out what *something bad* is. You will have to be careful about how you construct your trap test so it will handle alarm clearing and won't forget problems.
I am not throwing any rocks at Hobbit -- it was not designed to handle real-time alarm management. I *think* Henrik would agree...
With all that said, it is pretty easy to forward Hobbit alarms to Netcool, and I don't think it would be very hard to forward Netcool alarms to a Hobbit test -- if you are concerned about a dashboard, etc.
GLH
-----Original Message----- From: Jacobs, Eric (ThomasTech) [mailto:Eric.Jacobs at thomastechsolutions.com] Sent: Monday, July 16, 2007 5:05 PM To: 'hobbit at hswn.dk' Subject: RE: [hobbit] Hobbit not recognizing certain tests?
-----Original Message----- From: FARRIOR, Andy [mailto:Andy.Farrior at victoriacollege.edu] Sent: Monday, July 16, 2007 5:30 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Hobbit not recognizing certain tests?
Andy, wonderful to hear from you. And thanks for your method of getting snmp traps into Hobbit. It is a bit of a feather in my cap that I can do so. My boss is anxious to get rid of our highly priced NetCool monitoring package, but we needed to be able to monitor traps, at least from our F5 devices.
I haven't modified trap.pl in a very *long* time. Hmm... (I'm still using it.)
Going back over the code, I'm running hobbitdboard so I can get the validtime field for the test: "$BB $BBDISP \"hobbitdboard test=trap fields=hostname,validtime,color\" "
In case there's a host that doesn't send a trap that often, I wanted to change the color to green instead of having it go purple and generate an alert. So to get the time left for a test, I needed the hobbitdboard results. A side-effect is that the column won't appear until the device sends it's first trap.
Yes, I looked again at the code and realized that I was mistaken, and have already apologized to Henrik for having the temerity to suggest that there might be a bug in his code.
For the problem device, is the log from SEC showing a hostname or IP address for the device sending the trap?
Yes, but that was sometime ago. Maybe before I made some other corrections to get things working. I'm tempted to just down a server to cause a trap.
There may be a problem if it's sending a status message to Hobbit with
an IP address instead of a hostname. I don't think trap.pl does a good job (or at all) of trying to resolve an IP address to a hostname that's in bb-hosts.
Yes, I've seen that. In order to get things to work, I've had to make /etc/host file entries and make sure that the sending devices are configured to send their traps via certain interfaces so that the IP address will correspond to the host file address.
Hope that helps, Andy
-----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Monday, July 16, 2007 3:41 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Hobbit not recognizing certain tests?
On Mon, Jul 16, 2007 at 12:43:55PM -0400, Eric Jacobs wrote:
The trap.pl uses hobbitboard to find all the hosts that have the "trap" test defined.
Bad idea - there's a chicken-and-egg problem here.
Columns do not appear in the hobbitdboard output until they exist, i.e. a status has been reported to Hobbit. So if *something* must report a "trap" status to Hobbit before the trap.pl script sees that it should do the "trap" thing ... well, I guess you can tell where I'm
going.
Scripts that implement custom tests should use the "bbhostgrep" utility to scan the bb-hosts file for the hosts that have a specific tag defined.
There is actually another way it can be done, but it's not yet documented: You can send a "hostinfo" command to hobbitd, and it will return a pre-parsed version of the bb-hosts file - one line per hosts,
with the fields delimited by '|' characters, eg: mail.hswn.dk|172.16.10.2|NET:intern|COMMENT:Internal mail server|smtp|pop3|imap
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Tue, Jul 17, 2007 at 08:07:31AM -0500, Hubbard, Greg L wrote:
Being familiar with both Netcool and Hobbit, I would never consider replacing Netcool trap management with Hobbit. Netcool processes traps in "near real time" and scales quite large. Hobbit does not provide real time monitoring (default is 5 minute samples, with 1 minute screen updates).
There is nothing inherent in Hobbit that prevents it from doing real-time handling of events. Hobbit processes events as soon as it is told about them; the fact that some types of information is only checked once every 5 minutes is not something that necessarily applies to everything Hobbit monitors.
I havent looked at Andy's trap script, but if I were to implement SNMP trap handling in Hobbit, I'd start off with snmptrapd from the Net-SNMP tools - this receives snmp traps, and can be configured to do "something" when a trap arrives. That "something" would then be a script/utility that grabs the hostname and trap type from the trap information, and feeds that into Hobbit as a status update. That will give you an immediate alert, and a status change in the Hobbit display if you use the "Critical systems" view which is dynamically generated.
I'm not throwing rocks at Netcool <grin> but I just want to make it clear that Hobbit can be as real-time as you want it to - it's only a matter of feeding it data as quickly as you possible.
Regards, Henrik
We are on the same page then. I don't think I would ask Hobbit to do this for very many devices. I handle 100 traps per second in Netcool, and I am just getting started. But I love what Hobbit provides for monitoring the tool servers themselves -- without relying on anything on those same servers in order to function.
GLH
-----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Tuesday, July 17, 2007 8:33 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Hobbit not recognizing certain tests?
On Tue, Jul 17, 2007 at 08:07:31AM -0500, Hubbard, Greg L wrote:
Being familiar with both Netcool and Hobbit, I would never consider replacing Netcool trap management with Hobbit. Netcool processes traps in "near real time" and scales quite large. Hobbit does not provide real time monitoring (default is 5 minute samples, with 1 minute screen updates).
There is nothing inherent in Hobbit that prevents it from doing real-time handling of events. Hobbit processes events as soon as it is told about them; the fact that some types of information is only checked once every 5 minutes is not something that necessarily applies to everything Hobbit monitors.
I havent looked at Andy's trap script, but if I were to implement SNMP trap handling in Hobbit, I'd start off with snmptrapd from the Net-SNMP tools - this receives snmp traps, and can be configured to do "something" when a trap arrives. That "something" would then be a script/utility that grabs the hostname and trap type from the trap information, and feeds that into Hobbit as a status update. That will give you an immediate alert, and a status change in the Hobbit display if you use the "Critical systems" view which is dynamically generated.
I'm not throwing rocks at Netcool <grin> but I just want to make it clear that Hobbit can be as real-time as you want it to - it's only a matter of feeding it data as quickly as you possible.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On 7/17/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Tue, Jul 17, 2007 at 08:07:31AM -0500, Hubbard, Greg L wrote:
Being familiar with both Netcool and Hobbit, I would never consider replacing Netcool trap management with Hobbit. Netcool processes traps in "near real time" and scales quite large. Hobbit does not provide real time monitoring (default is 5 minute samples, with 1 minute screen updates).
There is nothing inherent in Hobbit that prevents it from doing real-time handling of events. Hobbit processes events as soon as it is told about them; the fact that some types of information is only checked once every 5 minutes is not something that necessarily applies to everything Hobbit monitors.
I havent looked at Andy's trap script, but if I were to implement SNMP trap handling in Hobbit, I'd start off with snmptrapd from the
I wish you do :-). I have lots of customers who like to see that feature in hobbit. May be then I can convince my IT department to get rid of Netcool slowly :P
Net-SNMP tools - this receives snmp traps, and can be configured to
do "something" when a trap arrives. That "something" would then be a script/utility that grabs the hostname and trap type from the trap information, and feeds that into Hobbit as a status update. That will give you an immediate alert, and a status change in the Hobbit display if you use the "Critical systems" view which is dynamically generated.
I'm not throwing rocks at Netcool <grin> but I just want to make it clear that Hobbit can be as real-time as you want it to - it's only a matter of feeding it data as quickly as you possible.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
Here's a quick overview of how I handle SNMP traps with Hobbit:
1 - Snmptrapd is configured to feed SNMPTT the OID and hostname of the sending SNMP agent.
2 - SNMPTT then translates the OID into a text message based on the MIB description for that OID and logs the message in /var/log/messages (or where ever you put it). You can also configure SNMPTT to log to a MySQL database simultaneously.
3 - SEC monitors the /var/log/messages file for entries from SNMPTT. Since some equipment can send the same trap multiple times in quick sucession, SEC is configured to ignore duplicate messages for a second or two
4 - SEC then launches a wrapper script that sends Hobbit a message using Hobbit's BB client program. Hobbit will send an alert if its status is yellow/red.
5 - A script is run by Hobbit every 5 minutes to prevent any trap message columns from turning purple. (I don't want my screen turning purple if I don't get a trap inside of 30min or whatever the no response timeout period is for Hobbit.)
In the event you get a rapid sequence of a "CRITICAL" trap and then a "Normal" trap, you'll get a Hobbit alert, but when you view the web page, it'll be green. You have to rely on the trap history to see all of the traps that SNMPTT recorded.
The real trick (pain) is defining what traps you want to be CRITICAL and what not.
Andy
From: Asif Iqbal [mailto:vadud3 at gmail.com] Sent: Tuesday, July 17, 2007 9:31 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Hobbit not recognizing certain tests?
On 7/17/07, Henrik Stoerner <henrik at hswn.dk> wrote:
On Tue, Jul 17, 2007 at 08:07:31AM -0500, Hubbard, Greg L wrote:
> Being familiar with both Netcool and Hobbit, I would neverconsider > replacing Netcool trap management with Hobbit. Netcool processes traps > in "near real time" and scales quite large. Hobbit does not provide > real time monitoring (default is 5 minute samples, with 1 minute screen > updates).
There is nothing inherent in Hobbit that prevents it from doing
real-time handling of events. Hobbit processes events as soon asit is told about them; the fact that some types of information is only checked once every 5 minutes is not something that necessarily applies to everything Hobbit monitors.
I havent looked at Andy's trap script, but if I were toimplement SNMP trap handling in Hobbit, I'd start off with snmptrapd from the
I wish you do :-). I have lots of customers who like to see that feature in hobbit. May be then I can convince my IT department to get rid of Netcool slowly :P
Net-SNMP tools - this receives snmp traps, and can be configuredto do "something" when a trap arrives. That "something" would then be a script/utility that grabs the hostname and trap type from the trap information, and feeds that into Hobbit as a status update. That will give you an immediate alert, and a status change in the Hobbit display if you use the "Critical systems" view which is dynamically generated.
I'm not throwing rocks at Netcool <grin> but I just want to makeit clear that Hobbit can be as real-time as you want it to - it's only a matter of feeding it data as quickly as you possible.
Regards,
Henrik
To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
participants (5)
-
Andy.Farrior＠victoriacollege.edu
-
Eric.Jacobs＠thomastechsolutions.com
-
greg.hubbard＠eds.com
-
henrik＠hswn.dk
-
vadud3＠gmail.com