-----Original Message----- From: Andy Smith [mailto:abs at shadymint.com] Sent: Wednesday, June 03, 2015 1:48 PM To: Larry Bonham Cc: xymon at xymon.com Subject: Re: [Xymon] Alternate log monitor?
Larry Bonham wrote:
Has anyone successfully integrated a different log monitor into xymon? The internal msgs monitor works fine for /var/log/messages or any other system log where a similar notification is used for each system (in our case the systems group).
But I have a need to monitor multiple Apache logs on various systems where a different user group would be notified if any yellow or red alerts were created. I don't see a way to do that and have /var/log/messages alerts going somewhere else for the same system.
Running 4.3.18 on RHEL 6.6
swatch maybe? Everything I'm finding is either too simplistic or way overkill for what I need. The lighter the footprint the better.
Thanks.
=========================================================
Larry D. Bonham
Hi, like you, we found the internal msgs tool is perfectly adequate in terms of functionality, but does not lend itself well to notifying different groups unless you are relying on email notification, in which case you can define a GROUP rule in alert.cfg/analysis.cfg. In our place, we do not use email, instead, we have a 24x7 operations team viewing the critical page, managing the incidents in real time, so we needed different column names to be able to identify the requirements in critical.cfg. I wrote this simple script to direct a given log analysis to a given column name, it may work for you :-
https://wiki.xymonton.org/doku.php/monitors:msgs
When we have need for more sophisticated log monitoring and we use SEC (http://simple-evcorr.sourceforge.net/) which is extremely powerful, we employ just a fraction of its potential. We have a perl module that goes with SEC so that SEC delivers status messages to Xymon in exactly the same way as the internal msgs tool.
Andy
Andy,
I appreciate the response. I should have mentioned I already tried your msgs.sh add on. Only problem is the original output stays in MSGS as well as getting copied to the alternate test name. We don't want the normal alerts going out to MSGS owners. Just to the owners of the alternate test name.
Unless I missed something in setting it up, I wasn't able to do that. I could do special notification on the alternate test but could not differentiate those in the normal test.
Please let me know if I'm misunderstanding something there (e.g. possible GROUP usage). Wouldn't be the first time.
I have used SEC in the past with snmp trap translation. That is one of those overkill solutions I mentioned. I'm sure it would work but I wanted to keep the solution as simple as possible for support purposes down the road. I was really hoping to accomplish this all within xymon itself if possible.
Thanks for your suggestions.
Larry
CONFIDENTIALITY NOTICE: This electronic mail message is intended exclusively for recipient to which it is addressed. The contents of this message and any attachments may contain confidential and privileged information. Any unauthorized review, use, print, storage, copy, disclosure or distribution is strictly prohibited. If you have received this message in error, please advise the sender immediately by replying to the message's sender and delete all copies of this message and its attachments without disclosing the contents to anyone, or using the contents for any purpose.
Larry Bonham wrote:
-----Original Message----- From: Andy Smith [mailto:abs at shadymint.com] Sent: Wednesday, June 03, 2015 1:48 PM To: Larry Bonham Cc: xymon at xymon.com Subject: Re: [Xymon] Alternate log monitor?
Larry Bonham wrote:
Has anyone successfully integrated a different log monitor into xymon? The internal msgs monitor works fine for /var/log/messages or any other system log where a similar notification is used for each system (in our case the systems group).
But I have a need to monitor multiple Apache logs on various systems where a different user group would be notified if any yellow or red alerts were created. I don't see a way to do that and have /var/log/messages alerts going somewhere else for the same system.
Running 4.3.18 on RHEL 6.6
swatch maybe? Everything I'm finding is either too simplistic or way overkill for what I need. The lighter the footprint the better.
Thanks.
=========================================================
Larry D. Bonham
Hi, like you, we found the internal msgs tool is perfectly adequate in terms of functionality, but does not lend itself well to notifying different groups unless you are relying on email notification, in which case you can define a GROUP rule in alert.cfg/analysis.cfg. In our place, we do not use email, instead, we have a 24x7 operations team viewing the critical page, managing the incidents in real time, so we needed different column names to be able to identify the requirements in critical.cfg. I wrote this simple script to direct a given log analysis to a given column name, it may work for you :-
https://wiki.xymonton.org/doku.php/monitors:msgs
When we have need for more sophisticated log monitoring and we use SEC (http://simple-evcorr.sourceforge.net/) which is extremely powerful, we employ just a fraction of its potential. We have a perl module that goes with SEC so that SEC delivers status messages to Xymon in exactly the same way as the internal msgs tool.
Andy
Andy,
I appreciate the response. I should have mentioned I already tried your msgs.sh add on. Only problem is the original output stays in MSGS as well as getting copied to the alternate test name. We don't want the normal alerts going out to MSGS owners. Just to the owners of the alternate test name.
Unless I missed something in setting it up, I wasn't able to do that. I could do special notification on the alternate test but could not differentiate those in the normal test.
Please let me know if I'm misunderstanding something there (e.g. possible GROUP usage). Wouldn't be the first time.
I have used SEC in the past with snmp trap translation. That is one of those overkill solutions I mentioned. I'm sure it would work but I wanted to keep the solution as simple as possible for support purposes down the road. I was really hoping to accomplish this all within xymon itself if possible.
Thanks for your suggestions.
Larry
For instance, on our apache web servers we suppress the msgs column completely (group-except msgs blah blah blah in hosts.cfg) and direct messages-ng to a new column called syslog which ultimately gets the the UNIX SA called out and the httpd error_log and access_log get directed to a column called weblog which gets our middleware team called out.
I would disagree that SEC is overkill. Some of our apache access logs are over 4GB per day, the best way we found to watch traffic in these volatile logs is with SEC. Admittedly, it is nothing more sophisticated than regex patterns we look for but there is no way we can afford to ship this amount of data to the central server for analysis. Yes, we generate some impact at the client side by analyzing locally instead of centrally, but extend this to tomcat logs and weblogic logs and get the power of rules like singlewithsuppress and contexts and we can easily find the needle in the haystack for our developers.
Andy
-----Original Message----- From: Andy Smith [mailto:abs at shadymint.com] Sent: Wednesday, June 03, 2015 3:37 PM To: Larry Bonham Cc: xymon at xymon.com Subject: Re: Alternate log monitor?
Larry Bonham wrote:
-----Original Message----- From: Andy Smith [mailto:abs at shadymint.com] Sent: Wednesday, June 03, 2015 1:48 PM To: Larry Bonham Cc: xymon at xymon.com Subject: Re: [Xymon] Alternate log monitor?
Larry Bonham wrote:
Has anyone successfully integrated a different log monitor into xymon? The internal msgs monitor works fine for /var/log/messages or any other system log where a similar notification is used for each system (in our case the systems group).
But I have a need to monitor multiple Apache logs on various systems where a different user group would be notified if any yellow or red alerts were created. I don't see a way to do that and have /var/log/messages alerts going somewhere else for the same system.
Running 4.3.18 on RHEL 6.6
swatch maybe? Everything I'm finding is either too simplistic or way overkill for what I need. The lighter the footprint the better.
Thanks.
=========================================================
Larry D. Bonham
Hi, like you, we found the internal msgs tool is perfectly adequate in terms of functionality, but does not lend itself well to notifying different groups unless you are relying on email notification, in which case you can define a GROUP rule in alert.cfg/analysis.cfg. In our place, we do not use email, instead, we have a 24x7 operations team viewing the critical page, managing the incidents in real time, so we needed different column names to be able to identify the requirements in critical.cfg. I wrote this simple script to direct a given log analysis to a given column name, it may work for you :-
https://wiki.xymonton.org/doku.php/monitors:msgs
When we have need for more sophisticated log monitoring and we use SEC (http://simple-evcorr.sourceforge.net/) which is extremely powerful, we employ just a fraction of its potential. We have a perl module that goes with SEC so that SEC delivers status messages to Xymon in exactly the same way as the internal msgs tool.
Andy
Andy,
I appreciate the response. I should have mentioned I already tried your msgs.sh add on. Only problem is the original output stays in MSGS as well as getting copied to the alternate test name. We don't want the normal alerts going out to MSGS owners. Just to the owners of the alternate test name.
Unless I missed something in setting it up, I wasn't able to do that. I could do special notification on the alternate test but could not differentiate those in the normal test.
Please let me know if I'm misunderstanding something there (e.g. possible GROUP usage). Wouldn't be the first time.
I have used SEC in the past with snmp trap translation. That is one of those overkill solutions I mentioned. I'm sure it would work but I wanted to keep the solution as simple as possible for support purposes down the road. I was really hoping to accomplish this all within xymon itself if possible.
Thanks for your suggestions.
Larry
For instance, on our apache web servers we suppress the msgs column completely (group-except msgs blah blah blah in hosts.cfg) and direct messages-ng to a new column called syslog which ultimately gets the the UNIX SA called out and the httpd error_log and access_log get directed to a column called weblog which gets our middleware team called out.
I would disagree that SEC is overkill. Some of our apache access logs are over 4GB per day, the best way we found to watch traffic in these volatile logs is with SEC. Admittedly, it is nothing more sophisticated than regex patterns we look for but there is no way we can afford to ship this amount of data to the central server for analysis. Yes, we generate some impact at the client side by analyzing locally instead of centrally, but extend this to tomcat logs and weblogic logs and get the power of rules like singlewithsuppress and contexts and we can easily find the needle in the haystack for our developers.
Andy
Thanks again Andy. Very good clarification on manipulating msgs.
Probably overkill was the wrong word to use on my part in regards to SEC. And I do plan on keeping the analysis on the client side. Only sending alerts or warnings to the central server.
Larry
CONFIDENTIALITY NOTICE: This electronic mail message is intended exclusively for recipient to which it is addressed. The contents of this message and any attachments may contain confidential and privileged information. Any unauthorized review, use, print, storage, copy, disclosure or distribution is strictly prohibited. If you have received this message in error, please advise the sender immediately by replying to the message's sender and delete all copies of this message and its attachments without disclosing the contents to anyone, or using the contents for any purpose.
participants (2)
-
abs＠shadymint.com
-
larry＠fni-stl.com