hobbitclient + msgs test sugesstion
Hello Henrik + fellow hobbiters,
Since the msgs-check is not available yet in the Hobbit display, I want to make a suggestion to have it enabled relatively easy. I think of two methods:
-1- A client must have read access to the file, lets say /var/adm/messages, but that has a major drawback: you can only monitor one [predefined] file. Anyway, you can pass on the major BB functionality. But any other logfiles, as (is/was) possible with bb-msgtab is discarded.
-2- Your Hobbit server must _also_ be a central loghost. In this way, all configuration remains on the Hobbit server (a GREAT welcome Hobbit-feature). You, Henrik, can even define the format that you want the central loghost being set up. Fi. seperate .info, .debug -> green, .kern -> yellow, etc.
Each host can use up a history of one week message-files and serious trouble can be set up/defined within, let's say, /etc/hobbit-client-messages.cfg. Now it is even possible to have this setup work for another logfile, let's say, /var/log/mail?
Perhaps (I am googling rigth now) this is useful: http://freshmeat.net/projects/syslog-ng/ but I guess this paints the picture.
Well, that's just my 2 cents.
Regards, Peter
Hi Peter (and anyone else interested),
On Tue, Nov 29, 2005 at 08:26:14PM +0100, Peter Welter wrote:
Since the msgs-check is not available yet in the Hobbit display, I want to make a suggestion to have it enabled relatively easy. I think of two methods:
-1- A client must have read access to the file [client picks out the "interesting" bits]
-2- Your Hobbit server must _also_ be a central loghost. [allows centralized configuration of how to monitor the logs]
I'm not really thrilled with either of these - sorry! Each of them have some drawbacks: The first one moves the configuration of what logs to monitor away from the central hobbit server, and the second one only works for logs that go through the syslog interface. If I want to monitor e.g. an Apache webserver error-log, or the custom logs from a BEA server, solution 2) won't work. I dont see how it can work with logs from a Windows server either. Plus it adds load to the central Hobbit server to deal with all of the logfiles.
So - I think some other solution is needed, and I've been thinking about how to do it. So far it's just ideas - no code. But I believe the log checking has to happen on each client, but controlled by a central configuration. So what I plan to implement is something like this:
- The configuration of what logs to monitor and what strings to look for is maintained on the central Hobbit server, either as an addition to the hobbit-clients.cfg file, or in a separate file - that isn't really important.
- When a client connects and sends in a client-side message, the Hobbit server accepts the client message, but also sends back the current log-check configuration info. By re-using the client connection, the overhead involved in pushing the configuration to each client becomes almost nil.
- When the client has a log-check configuration, it knows what logs to check for what strings, and can include that information in the normal client message it sends back to the Hobbit server. That means the client will need a tool to do the logfile checking; probably using a client-side regular-expression matching tool like "grep". It can either be built into the Hobbit client, or it could just rely on the existing "grep" utility found on the system - this would probably be the simplest to implement.
Regards, Henrik
Doesn't the current bb implementation only monitor log file messages issued during a time period just past (like 30 minutes)?
I like the expression matching idea, but repetitively walking through the whole thing could introduce alot of overhead. It would be beneficial walk backward from the end of the file to a pre-configured point in the past, then scan the messages. Sorry if this is obvious... :)
Henrik Stoerner wrote:
Hi Peter (and anyone else interested),
On Tue, Nov 29, 2005 at 08:26:14PM +0100, Peter Welter wrote:
Since the msgs-check is not available yet in the Hobbit display, I want to make a suggestion to have it enabled relatively easy. I think of two methods:
-1- A client must have read access to the file [client picks out the "interesting" bits]
-2- Your Hobbit server must _also_ be a central loghost. [allows centralized configuration of how to monitor the logs]
I'm not really thrilled with either of these - sorry! Each of them have some drawbacks: The first one moves the configuration of what logs to monitor away from the central hobbit server, and the second one only works for logs that go through the syslog interface. If I want to monitor e.g. an Apache webserver error-log, or the custom logs from a BEA server, solution 2) won't work. I dont see how it can work with logs from a Windows server either. Plus it adds load to the central Hobbit server to deal with all of the logfiles.
So - I think some other solution is needed, and I've been thinking about how to do it. So far it's just ideas - no code. But I believe the log checking has to happen on each client, but controlled by a central configuration. So what I plan to implement is something like this:
- The configuration of what logs to monitor and what strings to look for is maintained on the central Hobbit server, either as an addition to the hobbit-clients.cfg file, or in a separate file - that isn't really important.
- When a client connects and sends in a client-side message, the Hobbit server accepts the client message, but also sends back the current log-check configuration info. By re-using the client connection, the overhead involved in pushing the configuration to each client becomes almost nil.
- When the client has a log-check configuration, it knows what logs to check for what strings, and can include that information in the normal client message it sends back to the Hobbit server. That means the client will need a tool to do the logfile checking; probably using a client-side regular-expression matching tool like "grep". It can either be built into the Hobbit client, or it could just rely on the existing "grep" utility found on the system - this would probably be the simplest to implement.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Rich Smrcina VM Assist, Inc. Main: (262)392-2026 Cell: (414)491-6001 Ans Service: (360)715-2467 rich.smrcina at vmassist.com
Catch the WAVV! http://www.wavv.org WAVV 2006 - Chattanooga, TN - April 7-11, 2006
On Tue, Nov 29, 2005 at 10:09:48PM, Henrik Stoerner wrote:
Hi Peter (and anyone else interested),
On Tue, Nov 29, 2005 at 08:26:14PM +0100, Peter Welter wrote:
Since the msgs-check is not available yet in the Hobbit display, I want to make a suggestion to have it enabled relatively easy. I think of two methods:
-1- A client must have read access to the file [client picks out the "interesting" bits]
-2- Your Hobbit server must _also_ be a central loghost. [allows centralized configuration of how to monitor the logs]
I'm not really thrilled with either of these - sorry! Each of them have some drawbacks: The first one moves the configuration of what logs to monitor away from the central hobbit server, and the second one only works for logs that go through the syslog interface. If I want to monitor e.g. an Apache webserver error-log, or the custom logs from a BEA server, solution 2) won't work. I dont see how it can work with logs from a Windows server either. Plus it adds load to the central Hobbit server to deal with all of the logfiles.
So - I think some other solution is needed, and I've been thinking about how to do it. So far it's just ideas - no code. But I believe the log checking has to happen on each client, but controlled by a central configuration. So what I plan to implement is something like this:
- The configuration of what logs to monitor and what strings to look for is maintained on the central Hobbit server, either as an addition to the hobbit-clients.cfg file, or in a separate file - that isn't really important.
- When a client connects and sends in a client-side message, the Hobbit server accepts the client message, but also sends back the current log-check configuration info. By re-using the client connection, the overhead involved in pushing the configuration to each client becomes almost nil.
- When the client has a log-check configuration, it knows what logs to check for what strings, and can include that information in the normal client message it sends back to the Hobbit server. That means the client will need a tool to do the logfile checking; probably using a client-side regular-expression matching tool like "grep". It can either be built into the Hobbit client, or it could just rely on the existing "grep" utility found on the system - this would probably be the simplest to implement.
Would it be possible to create a new hobbitd channel that will get install with hobbit client. Then add that channel to the syslog.conf which is kind a work like a pipe. So when syslog say related to /var/adm/messages file get send to the hobbitd channel (or pipe) it will scan right away against strings that needs to get alerted about. Also it won't store anything in the channel. So there is no chance to scan the same string on the same timestamp twice. Also if it is not receiving any alert for say 5 mins it will check if syslogd is actually running by sending a 'logger' output to the channel.
Sorry if I talking 'no sense' but throwing anything here while the idea is still cooking :-)
Thanks
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu "..there are two kinds of people: those who work and those who take the credit...try to be in the first group;...less competition there." - Indira Gandhi
Hi,
Why should the log-entries, hobbit-msg-monitor should look after be maintained centrally on the hobbit server? Are important log-entries essential on a sql-server as well in every case as on a firewall? I think that there are different things on different servers you want hobbit to take care about... Maybe I missed the point...
To reduce overhead, you can use a similar mechanism as logtail does. Storing the file offset in conjunction with the inode-id would grant you never check an entry twice.
Maybe, having a closer look at logsentry from the sentry-tools (http://sourceforge.net/projects/sentrytools) would help finding an appropriate way of realizing this.
Kind regards,
Manuel
----- Message from iqbala-hobbit at qwestip.net --------- Date: Tue, 29 Nov 2005 16:42:21 -0500 From: Asif Iqbal <iqbala-hobbit at qwestip.net> Reply-To: hobbit at hswn.dk Subject: Re: [hobbit] hobbitclient + msgs test sugesstion To: hobbit at hswn.dk
On Tue, Nov 29, 2005 at 10:09:48PM, Henrik Stoerner wrote:
Hi Peter (and anyone else interested),
On Tue, Nov 29, 2005 at 08:26:14PM +0100, Peter Welter wrote:
Since the msgs-check is not available yet in the Hobbit display, I want to make a suggestion to have it enabled relatively easy. I think of two methods:
-1- A client must have read access to the file [client picks out the "interesting" bits]
-2- Your Hobbit server must _also_ be a central loghost. [allows centralized configuration of how to monitor the logs]
I'm not really thrilled with either of these - sorry! Each of them have some drawbacks: The first one moves the configuration of what logs to monitor away from the central hobbit server, and the second one only works for logs that go through the syslog interface. If I want to monitor e.g. an Apache webserver error-log, or the custom logs from a BEA server, solution 2) won't work. I dont see how it can work with logs from a Windows server either. Plus it adds load to the central Hobbit server to deal with all of the logfiles.
So - I think some other solution is needed, and I've been thinking about how to do it. So far it's just ideas - no code. But I believe the log checking has to happen on each client, but controlled by a central configuration. So what I plan to implement is something like this:
- The configuration of what logs to monitor and what strings to look for is maintained on the central Hobbit server, either as an addition to the hobbit-clients.cfg file, or in a separate file - that isn't really important.
- When a client connects and sends in a client-side message, the Hobbit server accepts the client message, but also sends back the current log-check configuration info. By re-using the client connection, the overhead involved in pushing the configuration to each client becomes almost nil.
- When the client has a log-check configuration, it knows what logs to check for what strings, and can include that information in the normal client message it sends back to the Hobbit server. That means the client will need a tool to do the logfile checking; probably using a client-side regular-expression matching tool like "grep". It can either be built into the Hobbit client, or it could just rely on the existing "grep" utility found on the system - this would probably be the simplest to implement.
Would it be possible to create a new hobbitd channel that will get install with hobbit client. Then add that channel to the syslog.conf which is kind a work like a pipe. So when syslog say related to /var/adm/messages file get send to the hobbitd channel (or pipe) it will scan right away against strings that needs to get alerted about. Also it won't store anything in the channel. So there is no chance to scan the same string on the same timestamp twice. Also if it is not receiving any alert for say 5 mins it will check if syslogd is actually running by sending a 'logger' output to the channel.
Sorry if I talking 'no sense' but throwing anything here while the idea is still cooking :-)
Thanks
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu "..there are two kinds of people: those who work and those who take the credit...try to be in the first group;...less competition there." - Indira Gandhi
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
----- End message from iqbala-hobbit at qwestip.net -----
A log file monitor would also need to consider what happens when logrotate (or other switching mechanism) switches logs, as the inode and file offset may (or will) change. I know some people have a cron job which copies the log, then truncates the original log file. In this case the inode won't change, but the file contents / offset will. There are many things to consider, I'm sure Henrik will come up with a good solution.
Manu <yoogie at schurkennetz.de> 11/29/2005 05:38 PM Please respond to hobbit at hswn.dk
To hobbit at hswn.dk cc
Subject Re: [hobbit] hobbitclient + msgs test sugesstion
Hi,
Why should the log-entries, hobbit-msg-monitor should look after be maintained centrally on the hobbit server? Are important log-entries essential on a sql-server as well in every case as on a firewall? I think that there are different things on different servers you want hobbit to take care about... Maybe I missed the point...
To reduce overhead, you can use a similar mechanism as logtail does. Storing the file offset in conjunction with the inode-id would grant you never check an entry twice.
Maybe, having a closer look at logsentry from the sentry-tools (http://sourceforge.net/projects/sentrytools) would help finding an appropriate way of realizing this.
Kind regards,
Manuel
----- Message from iqbala-hobbit at qwestip.net --------- Date: Tue, 29 Nov 2005 16:42:21 -0500 From: Asif Iqbal <iqbala-hobbit at qwestip.net> Reply-To: hobbit at hswn.dk Subject: Re: [hobbit] hobbitclient + msgs test sugesstion To: hobbit at hswn.dk
On Tue, Nov 29, 2005 at 10:09:48PM, Henrik Stoerner wrote:
Hi Peter (and anyone else interested),
On Tue, Nov 29, 2005 at 08:26:14PM +0100, Peter Welter wrote:
Since the msgs-check is not available yet in the Hobbit display, I want to make a suggestion to have it enabled relatively easy. I think of two methods:
-1- A client must have read access to the file [client picks out the "interesting" bits]
-2- Your Hobbit server must _also_ be a central loghost. [allows centralized configuration of how to monitor the logs]
I'm not really thrilled with either of these - sorry! Each of them have some drawbacks: The first one moves the configuration of what logs to monitor away from the central hobbit server, and the second one only works for logs that go through the syslog interface. If I want to monitor e.g. an Apache webserver error-log, or the custom logs from a BEA server, solution 2) won't work. I dont see how it can work with logs from a Windows server either. Plus it adds load to the central Hobbit server to deal with all of the logfiles.
So - I think some other solution is needed, and I've been thinking about how to do it. So far it's just ideas - no code. But I believe the log checking has to happen on each client, but controlled by a central configuration. So what I plan to implement is something like this:
- The configuration of what logs to monitor and what strings to look for is maintained on the central Hobbit server, either as an addition to the hobbit-clients.cfg file, or in a separate file - that isn't really important.
- When a client connects and sends in a client-side message, the Hobbit server accepts the client message, but also sends back the current log-check configuration info. By re-using the client connection, the overhead involved in pushing the configuration to each client becomes almost nil.
- When the client has a log-check configuration, it knows what logs to check for what strings, and can include that information in the normal client message it sends back to the Hobbit server. That means the client will need a tool to do the logfile checking; probably using a client-side regular-expression matching tool like "grep". It can either be built into the Hobbit client, or it could just rely on the existing "grep" utility found on the system - this would probably be the simplest to implement.
Would it be possible to create a new hobbitd channel that will get install with hobbit client. Then add that channel to the syslog.conf which is kind a work like a pipe. So when syslog say related to /var/adm/messages file get send to the hobbitd channel (or pipe) it will scan right away against strings that needs to get alerted about. Also it won't store anything in the channel. So there is no chance to scan the same string on the same timestamp twice. Also if it is not receiving any alert for say 5 mins it will check if syslogd is actually running by sending a 'logger' output to the channel.
Sorry if I talking 'no sense' but throwing anything here while the idea is still cooking :-)
Thanks
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu "..there are two kinds of people: those who work and those who take the credit...try to be in the first group;...less competition there." - Indira Gandhi
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
----- End message from iqbala-hobbit at qwestip.net -----
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
participants (6)
-
Allan.Marillier＠dana.com
-
henrik＠hswn.dk
-
iqbala-hobbit＠qwestip.net
-
peter.welter＠gmail.com
-
rsmrcina＠wi.rr.com
-
yoogie＠schurkennetz.de